[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:00 dist-test-slave-sb2z krb5kdc[13472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:00 dist-test-slave-sb2z krb5kdc[13472](info): set up 2 sockets
Aug 12 17:48:00 dist-test-slave-sb2z krb5kdc[13472](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 17:48:01.905829 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.410s user 0.005s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:02 dist-test-slave-sb2z krb5kdc[13472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:48:02.056493 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:44811
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:44811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:02.187577 13493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:02.187918 13493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:02.187963 13493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:02.192363 13493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:02.192441 13493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:02.192462 13493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:02.192480 13493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:02.192497 13493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:02.198551 13493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:44811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:44811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:02.200132 13493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:02.201474 13493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:02.214206 13501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:02.214351 13493 server_base.cc:1034] running on GCE node
W20260812 17:48:02.215003 13505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:02.215368 13500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:02.216291 13493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:02.218014 13493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:02.219240 13493 hybrid_clock.cc:648] HybridClock initialized: now 1786556882219202 us; error 84 us; skew 500 ppm
Aug 12 17:48:02 dist-test-slave-sb2z krb5kdc[13472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556882, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:02.222661 13493 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:48:02.224213 13493 webserver.cc:460] Webserver started at http://127.13.38.62:32853/ using document root <none> and password file <none>
I20260812 17:48:02.224884 13493 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:02.224952 13493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:02.225260 13493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:02.228183 13493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "f369dc5641be4c5da003e81ad6bda12a"
format_stamp: "Formatted at 2026-08-12 17:48:02 on dist-test-slave-sb2z"
server_key: "406e6f82dbae4aaa2c6c2f5805756f0f"
server_key_iv: "27b4409744febb20119a824068cab0b0"
server_key_version: "encryptionkey@0"
I20260812 17:48:02.229146 13493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "f369dc5641be4c5da003e81ad6bda12a"
format_stamp: "Formatted at 2026-08-12 17:48:02 on dist-test-slave-sb2z"
server_key: "406e6f82dbae4aaa2c6c2f5805756f0f"
server_key_iv: "27b4409744febb20119a824068cab0b0"
server_key_version: "encryptionkey@0"
I20260812 17:48:02.234449 13493 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 17:48:02.238560 13512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:02.240263 13493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:02.240442 13493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "f369dc5641be4c5da003e81ad6bda12a"
format_stamp: "Formatted at 2026-08-12 17:48:02 on dist-test-slave-sb2z"
server_key: "406e6f82dbae4aaa2c6c2f5805756f0f"
server_key_iv: "27b4409744febb20119a824068cab0b0"
server_key_version: "encryptionkey@0"
I20260812 17:48:02.240588 13493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:02.255757 13493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:02.259464 13493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:02.259744 13493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:02.272605 13493 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:44811
I20260812 17:48:02.272722 13590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:44811 every 8 connection(s)
I20260812 17:48:02.274070 13493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:02.275692 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 13493
I20260812 17:48:02.275873 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:02.277649 13464 external_mini_cluster.cc:1428] Setting key 6a4445a8f1846080064605722f5f4525
I20260812 17:48:02.280548 13591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:02.290597 13591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a: Bootstrap starting.
I20260812 17:48:02.294600 13591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:02.295919 13591 log.cc:824] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a: Log is configured to *not* fsync() on all Append() calls
Aug 12 17:48:02 dist-test-slave-sb2z krb5kdc[13472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:02.298799 13591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a: No bootstrap required, opened a new log
I20260812 17:48:02.303833 13591 raft_consensus.cc:348] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f369dc5641be4c5da003e81ad6bda12a" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44811 } }
I20260812 17:48:02.304294 13591 raft_consensus.cc:374] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:02.304422 13591 raft_consensus.cc:729] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f369dc5641be4c5da003e81ad6bda12a, State: Initialized, Role: FOLLOWER
I20260812 17:48:02.304769 13594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:02.282714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.13.38.62:44811)
0812 17:48:02.283239 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:02.283263 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:02.284673 (+ 1410us) server_negotiation.cc:407] Connection header received
0812 17:48:02.287954 (+ 3281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:02.287973 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:02.288366 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:02.288707 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:02.291219 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:02.292632 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:02.293985 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:02.294530 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:02.294747 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:02.298895 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:02.298937 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:02.298966 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:02.299008 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:02.300834 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:02.301419 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:02.301425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:02.301429 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:02.301542 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:02.301901 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:02.301905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:02.301907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:02.302409 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:02.302659 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:02.303290 (+ 631us) server_negotiation.cc:299] Negotiation successful
0812 17:48:02.303628 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":175,"threads_started":1}
I20260812 17:48:02.305291 13591 consensus_queue.cc:260] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f369dc5641be4c5da003e81ad6bda12a" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44811 } }
I20260812 17:48:02.305512 13591 raft_consensus.cc:388] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:02.305649 13591 raft_consensus.cc:482] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:02.305774 13591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:02.306774 13591 raft_consensus.cc:504] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f369dc5641be4c5da003e81ad6bda12a" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44811 } }
I20260812 17:48:02.307402 13591 leader_election.cc:302] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f369dc5641be4c5da003e81ad6bda12a; no voters:
I20260812 17:48:02.307876 13591 leader_election.cc:288] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:02.308019 13597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:02.308239 13597 raft_consensus.cc:686] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [term 1 LEADER]: Becoming Leader. State: Replica: f369dc5641be4c5da003e81ad6bda12a, State: Running, Role: LEADER
I20260812 17:48:02.308727 13597 consensus_queue.cc:237] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f369dc5641be4c5da003e81ad6bda12a" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44811 } }
I20260812 17:48:02.309890 13591 sys_catalog.cc:564] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:02.310745 13597 sys_catalog.cc:455] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f369dc5641be4c5da003e81ad6bda12a. Latest consensus state: current_term: 1 leader_uuid: "f369dc5641be4c5da003e81ad6bda12a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f369dc5641be4c5da003e81ad6bda12a" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44811 } } }
I20260812 17:48:02.310873 13597 sys_catalog.cc:458] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:02.312259 13598 sys_catalog.cc:455] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f369dc5641be4c5da003e81ad6bda12a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f369dc5641be4c5da003e81ad6bda12a" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44811 } } }
I20260812 17:48:02.312405 13598 sys_catalog.cc:458] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:02.312983 13608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:02.318650 13608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:02.328238 13608 catalog_manager.cc:1347] Generated new cluster ID: 0d88037df40247c6aeeb1d2447d46a4c
I20260812 17:48:02.328384 13608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:02.348079 13608 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:02.349656 13608 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:02.355760 13608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f369dc5641be4c5da003e81ad6bda12a: Generated new TSK 0
I20260812 17:48:02.356886 13608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:48:02.396194 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44811
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:02.528402 13632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:02.528844 13632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:02.528980 13632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:02.533234 13632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:02.533336 13632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:02.533438 13632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:02.539800 13632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44811
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:02.541126 13632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:02.542250 13632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:02.553588 13642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:02.553603 13640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:02.553622 13648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:02.554179 13632 server_base.cc:1034] running on GCE node
I20260812 17:48:02.554646 13632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:02.555362 13632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:02.556573 13632 hybrid_clock.cc:648] HybridClock initialized: now 1786556882556549 us; error 46 us; skew 500 ppm
Aug 12 17:48:02 dist-test-slave-sb2z krb5kdc[13472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556882, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:02.560112 13632 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:48:02.561162 13632 webserver.cc:460] Webserver started at http://127.13.38.1:40955/ using document root <none> and password file <none>
I20260812 17:48:02.561774 13632 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:02.561869 13632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:02.562119 13632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:02.564478 13632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "c1b6731b85bd4bbb94a4db5ab13cef4b"
format_stamp: "Formatted at 2026-08-12 17:48:02 on dist-test-slave-sb2z"
server_key: "6bceae46ac91a98dbbe650c963344895"
server_key_iv: "5e6a554b7bfa3e06a400926ac7185464"
server_key_version: "encryptionkey@0"
I20260812 17:48:02.565141 13632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "c1b6731b85bd4bbb94a4db5ab13cef4b"
format_stamp: "Formatted at 2026-08-12 17:48:02 on dist-test-slave-sb2z"
server_key: "6bceae46ac91a98dbbe650c963344895"
server_key_iv: "5e6a554b7bfa3e06a400926ac7185464"
server_key_version: "encryptionkey@0"
I20260812 17:48:02.569514 13632 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 17:48:02.572232 13657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:02.573496 13632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:02.573660 13632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "c1b6731b85bd4bbb94a4db5ab13cef4b"
format_stamp: "Formatted at 2026-08-12 17:48:02 on dist-test-slave-sb2z"
server_key: "6bceae46ac91a98dbbe650c963344895"
server_key_iv: "5e6a554b7bfa3e06a400926ac7185464"
server_key_version: "encryptionkey@0"
I20260812 17:48:02.573791 13632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:02.598769 13632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:02.602542 13632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:02.602751 13632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:02.603667 13632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:02.604820 13632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:02.604876 13632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:02.604930 13632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:02.604954 13632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:02.619292 13632 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:38437
I20260812 17:48:02.619320 13843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:38437 every 8 connection(s)
I20260812 17:48:02.620438 13632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:02.625322 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 13632
I20260812 17:48:02.625432 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:02.625641 13464 external_mini_cluster.cc:1428] Setting key 41e4846c86bb83a791cc7ae3491e62bf
Aug 12 17:48:02 dist-test-slave-sb2z krb5kdc[13472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556882, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:48:02.638273 13594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:02.622995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:43657 (local address 127.13.38.62:44811)
0812 17:48:02.623158 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:02.623163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:02.624210 (+ 1047us) server_negotiation.cc:407] Connection header received
0812 17:48:02.625982 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:02.625986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:02.626046 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:02.626137 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:02.627763 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:02.628609 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:02.629894 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:02.630323 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:02.630470 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:02.633824 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:02.633827 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:02.633829 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:02.633855 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:02.635521 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:02.636356 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:02.636360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:02.636361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:02.636464 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:02.636884 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:02.636887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:02.636888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:02.637094 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:02.637175 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:02.637985 (+ 810us) server_negotiation.cc:299] Negotiation successful
0812 17:48:02.638136 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 17:48:02.639300 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:02.623405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44811 (local address 127.13.38.1:43657)
0812 17:48:02.624035 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:02.624092 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:02.625704 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:02.626379 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:02.626394 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:02.626926 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:02.627604 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:02.627626 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:02.628760 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:02.628763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:02.629754 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:02.629765 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:02.630518 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:02.630522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:02.630676 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:02.631437 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:02.631455 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:02.633608 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:02.635725 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:02.635734 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:02.635757 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:02.636205 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:02.636567 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:02.636571 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:02.636573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:02.636774 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:02.637218 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:02.637229 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:02.637615 (+ 386us) client_negotiation.cc:769] Sending connection context
0812 17:48:02.637842 (+ 227us) client_negotiation.cc:240] Negotiation successful
0812 17:48:02.638255 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":171,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
I20260812 17:48:02.640972 13844 heartbeater.cc:347] Connected to a master server at 127.13.38.62:44811
I20260812 17:48:02.641268 13844 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:02.641814 13844 heartbeater.cc:511] Master 127.13.38.62:44811 requested a full tablet report, sending...
I20260812 17:48:02.644223 13538 ts_manager.cc:194] Registered new tserver with Master: c1b6731b85bd4bbb94a4db5ab13cef4b (127.13.38.1:38437)
I20260812 17:48:02.645898 13538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:43657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:48:02.659009 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44811
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:02.788506 13853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:02.788803 13853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:02.788851 13853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:02.793084 13853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:02.793159 13853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:02.793251 13853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:02.799082 13853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44811
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:02.800292 13853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:02.801417 13853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:02.812469 13861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:02.812546 13860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:02.812490 13863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:02.813503 13853 server_base.cc:1034] running on GCE node
I20260812 17:48:02.813906 13853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:02.814560 13853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:02.815757 13853 hybrid_clock.cc:648] HybridClock initialized: now 1786556882815727 us; error 62 us; skew 500 ppm
Aug 12 17:48:02 dist-test-slave-sb2z krb5kdc[13472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556882, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:02.818684 13853 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:48:02.819849 13853 webserver.cc:460] Webserver started at http://127.13.38.2:37435/ using document root <none> and password file <none>
I20260812 17:48:02.820468 13853 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:02.820565 13853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:02.820816 13853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:02.823176 13853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "e9ac7606da9647db9bfca8b72e398f3f"
format_stamp: "Formatted at 2026-08-12 17:48:02 on dist-test-slave-sb2z"
server_key: "a791ef4d97bcbe1955086e9ebcf57f36"
server_key_iv: "ca4f4f853819e9f57d9c80179a6afa94"
server_key_version: "encryptionkey@0"
I20260812 17:48:02.823896 13853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "e9ac7606da9647db9bfca8b72e398f3f"
format_stamp: "Formatted at 2026-08-12 17:48:02 on dist-test-slave-sb2z"
server_key: "a791ef4d97bcbe1955086e9ebcf57f36"
server_key_iv: "ca4f4f853819e9f57d9c80179a6afa94"
server_key_version: "encryptionkey@0"
I20260812 17:48:02.828437 13853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 17:48:02.831272 13872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:02.832692 13853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:48:02.832854 13853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "e9ac7606da9647db9bfca8b72e398f3f"
format_stamp: "Formatted at 2026-08-12 17:48:02 on dist-test-slave-sb2z"
server_key: "a791ef4d97bcbe1955086e9ebcf57f36"
server_key_iv: "ca4f4f853819e9f57d9c80179a6afa94"
server_key_version: "encryptionkey@0"
I20260812 17:48:02.832996 13853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:02.847766 13853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:02.850970 13853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:02.851220 13853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:02.851974 13853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:02.852870 13853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:02.852946 13853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:02.853021 13853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:02.853075 13853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:02.865792 13853 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:39999
I20260812 17:48:02.865811 14043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:39999 every 8 connection(s)
I20260812 17:48:02.866861 13853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:02.867743 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 13853
I20260812 17:48:02.867880 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:02.868242 13464 external_mini_cluster.cc:1428] Setting key 8dbbc567bd9694337f2244b496df551c
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
Aug 12 17:48:02 dist-test-slave-sb2z krb5kdc[13472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556882, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:02.890731 13594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:02.869283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:47273 (local address 127.13.38.62:44811)
0812 17:48:02.869446 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:02.869451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:02.873032 (+ 3581us) server_negotiation.cc:407] Connection header received
0812 17:48:02.874182 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:02.874185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:02.874252 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:02.874351 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:02.875959 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:02.876907 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:02.878470 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:02.878874 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:02.879016 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:02.885341 (+ 6325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:02.885348 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:02.885363 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:02.885443 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:02.887726 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:02.888590 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:02.888594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:02.888595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:02.888696 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:02.889176 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:02.889180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:02.889181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:02.889404 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:02.889507 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:02.890451 (+ 944us) server_negotiation.cc:299] Negotiation successful
0812 17:48:02.890593 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 17:48:02.892282 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:02.869796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44811 (local address 127.13.38.2:47273)
0812 17:48:02.872848 (+ 3052us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:02.872903 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:02.873926 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:02.874563 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:02.874572 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:02.875089 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:02.875756 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:02.875777 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:02.877080 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:02.877083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:02.878265 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:02.878280 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:02.879095 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:02.879098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:02.879276 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:02.880102 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:02.880122 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:02.885044 (+ 4922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:02.887887 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:02.887898 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:02.887924 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:02.888426 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:02.888810 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:02.888813 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:02.888815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:02.889046 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:02.889650 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:02.889663 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:02.890069 (+ 406us) client_negotiation.cc:769] Sending connection context
0812 17:48:02.890321 (+ 252us) client_negotiation.cc:240] Negotiation successful
0812 17:48:02.890878 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2804,"thread_start_us":727,"threads_started":1}
I20260812 17:48:02.893884 14046 heartbeater.cc:347] Connected to a master server at 127.13.38.62:44811
I20260812 17:48:02.894250 14046 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:02.894826 14046 heartbeater.cc:511] Master 127.13.38.62:44811 requested a full tablet report, sending...
I20260812 17:48:02.896127 13538 ts_manager.cc:194] Registered new tserver with Master: e9ac7606da9647db9bfca8b72e398f3f (127.13.38.2:39999)
I20260812 17:48:02.896752 13538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:47273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:48:02.907815 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44811
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:03.039757 14052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:03.040063 14052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:03.040112 14052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:03.044245 14052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:03.044353 14052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:03.044449 14052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:03.050420 14052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44811
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:03.051657 14052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:03.052748 14052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:03.063057 14059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:03.063131 14061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:03.063057 14063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:03.063565 14052 server_base.cc:1034] running on GCE node
I20260812 17:48:03.064091 14052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:03.064798 14052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:03.066004 14052 hybrid_clock.cc:648] HybridClock initialized: now 1786556883065973 us; error 37 us; skew 500 ppm
Aug 12 17:48:03 dist-test-slave-sb2z krb5kdc[13472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556883, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:03.069238 14052 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:48:03.070264 14052 webserver.cc:460] Webserver started at http://127.13.38.3:36219/ using document root <none> and password file <none>
I20260812 17:48:03.070885 14052 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:03.070981 14052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:03.071239 14052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:03.073530 14052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "194bcc2b4ebd484d8a6a94114396a4d0"
format_stamp: "Formatted at 2026-08-12 17:48:03 on dist-test-slave-sb2z"
server_key: "d6c8eb32662a0db5951d54418310b58d"
server_key_iv: "ed9221f0207477ab89d8af39a005de1b"
server_key_version: "encryptionkey@0"
I20260812 17:48:03.074164 14052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "194bcc2b4ebd484d8a6a94114396a4d0"
format_stamp: "Formatted at 2026-08-12 17:48:03 on dist-test-slave-sb2z"
server_key: "d6c8eb32662a0db5951d54418310b58d"
server_key_iv: "ed9221f0207477ab89d8af39a005de1b"
server_key_version: "encryptionkey@0"
I20260812 17:48:03.078652 14052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 17:48:03.081423 14074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:03.082753 14052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 17:48:03.082898 14052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "194bcc2b4ebd484d8a6a94114396a4d0"
format_stamp: "Formatted at 2026-08-12 17:48:03 on dist-test-slave-sb2z"
server_key: "d6c8eb32662a0db5951d54418310b58d"
server_key_iv: "ed9221f0207477ab89d8af39a005de1b"
server_key_version: "encryptionkey@0"
I20260812 17:48:03.083031 14052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:03.110358 14052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:03.114149 14052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:03.114393 14052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:03.115140 14052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:03.116111 14052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:03.116194 14052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:03.116273 14052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:03.116322 14052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:03.128692 14052 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:45621
I20260812 17:48:03.128712 14259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:45621 every 8 connection(s)
I20260812 17:48:03.129686 14052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:03.137713 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 14052
I20260812 17:48:03.137843 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:03.138136 13464 external_mini_cluster.cc:1428] Setting key fce2c1184c00279fbf377e6ba93a9fa7
Aug 12 17:48:03 dist-test-slave-sb2z krb5kdc[13472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556883, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:03.145872 13594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:03.131961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:45031 (local address 127.13.38.62:44811)
0812 17:48:03.132114 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:03.132118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:03.133057 (+ 939us) server_negotiation.cc:407] Connection header received
0812 17:48:03.134450 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:03.134454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:03.134516 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:03.134631 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:03.136036 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:03.136631 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:03.138060 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:03.138368 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:03.138489 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:03.141890 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:03.141894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:03.141902 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:03.141928 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:03.143268 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:03.143992 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:03.143998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:03.144000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:03.144099 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:03.144578 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:03.144582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:03.144583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:03.144796 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:03.144887 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:03.145561 (+ 674us) server_negotiation.cc:299] Negotiation successful
0812 17:48:03.145690 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:48:03.146806 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:03.132403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44811 (local address 127.13.38.3:45031)
0812 17:48:03.132895 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:03.132941 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:03.134214 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:03.134836 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:03.134844 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:03.135252 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:03.135865 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:03.135882 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:03.136772 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:03.136781 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:03.137922 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:03.137940 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:03.138541 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:03.138546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:03.138737 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:03.139862 (+ 1125us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:03.139887 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:03.141721 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:03.143431 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:03.143436 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:03.143450 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:03.143827 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:03.144241 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:03.144244 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:03.144245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:03.144436 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:03.144933 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:03.144941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:03.145219 (+ 278us) client_negotiation.cc:769] Sending connection context
0812 17:48:03.145485 (+ 266us) client_negotiation.cc:240] Negotiation successful
0812 17:48:03.145864 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":113,"threads_started":1}
I20260812 17:48:03.148103 14260 heartbeater.cc:347] Connected to a master server at 127.13.38.62:44811
I20260812 17:48:03.148342 14260 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:03.148849 14260 heartbeater.cc:511] Master 127.13.38.62:44811 requested a full tablet report, sending...
I20260812 17:48:03.149899 13538 ts_manager.cc:194] Registered new tserver with Master: 194bcc2b4ebd484d8a6a94114396a4d0 (127.13.38.3:45621)
I20260812 17:48:03.150480 13538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:45031
I20260812 17:48:03.153339 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 17:48:03 dist-test-slave-sb2z krb5kdc[13472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556883, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 17:48:03 dist-test-slave-sb2z krb5kdc[13472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556883, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.38.1@KRBTEST.COM
I20260812 17:48:03.185650 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:03.174169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.13.38.1:38437)
0812 17:48:03.174619 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:03.174630 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:03.174660 (+ 30us) server_negotiation.cc:407] Connection header received
0812 17:48:03.174854 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:03.174860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:03.175081 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:03.175249 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:03.176076 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:03.177182 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:03.178132 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:03.178610 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:03.178730 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:03.181463 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:03.181473 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:03.181488 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:03.181517 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:03.183367 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:03.183936 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:03.183942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:03.183943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:03.184045 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:03.184407 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:03.184409 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:03.184411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:03.184688 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:03.184875 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:03.185257 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 17:48:03.185477 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":136,"threads_started":1}
W20260812 17:48:03.186859 13708 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40462
Aug 12 17:48:03 dist-test-slave-sb2z krb5kdc[13472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 17:48:03 dist-test-slave-sb2z krb5kdc[13472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.1@KRBTEST.COM
I20260812 17:48:03.217505 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:03.207977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40476 (local address 127.13.38.1:38437)
0812 17:48:03.208199 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:03.208204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:03.208275 (+ 71us) server_negotiation.cc:407] Connection header received
0812 17:48:03.208413 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:03.208418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:03.208487 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:03.208589 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:03.209304 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:03.209923 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:03.211027 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:03.211377 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:03.211497 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:03.214094 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:03.214099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:03.214100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:03.214127 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:03.215788 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:03.216262 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:03.216264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:03.216265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:03.216343 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:03.216584 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:03.216586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:03.216587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:03.216755 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:03.216857 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:03.217114 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 17:48:03.217295 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 17:48:03.218797 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 13632
I20260812 17:48:03.244473 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 13853
I20260812 17:48:03.259402 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 14052
I20260812 17:48:03.272101 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 13493
2026-08-12T17:48:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2822 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:03 dist-test-slave-sb2z krb5kdc[14286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:03 dist-test-slave-sb2z krb5kdc[14286](info): set up 2 sockets
Aug 12 17:48:03 dist-test-slave-sb2z krb5kdc[14286](info): commencing operation
krb5kdc: starting...
W20260812 17:48:04.561718 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:04 dist-test-slave-sb2z krb5kdc[14286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:48:04.713120 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:46423
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:46423
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:04.832897 14307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:04.833189 14307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:04.833233 14307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:04.837430 14307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:04.837509 14307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:04.837530 14307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:04.837548 14307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:04.837563 14307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:04.843616 14307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:46423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:46423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:04.844815 14307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:04.845901 14307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:04.854650 14318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:04.854664 14313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:04.854866 14314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:04.855023 14307 server_base.cc:1034] running on GCE node
I20260812 17:48:04.855729 14307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:04.856890 14307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:04.858086 14307 hybrid_clock.cc:648] HybridClock initialized: now 1786556884858056 us; error 45 us; skew 500 ppm
Aug 12 17:48:04 dist-test-slave-sb2z krb5kdc[14286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556884, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:04.861148 14307 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:48:04.862062 14307 webserver.cc:460] Webserver started at http://127.13.38.62:37239/ using document root <none> and password file <none>
I20260812 17:48:04.862598 14307 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:04.862658 14307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:04.862820 14307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:04.864933 14307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "ce4ceb9431b844d9a93d2e5af8e75d0f"
format_stamp: "Formatted at 2026-08-12 17:48:04 on dist-test-slave-sb2z"
server_key: "f43bd827b46b17c6ff987ae77185e068"
server_key_iv: "21092542bbb4d6802bc3c7ddc31796f2"
server_key_version: "encryptionkey@0"
I20260812 17:48:04.865491 14307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "ce4ceb9431b844d9a93d2e5af8e75d0f"
format_stamp: "Formatted at 2026-08-12 17:48:04 on dist-test-slave-sb2z"
server_key: "f43bd827b46b17c6ff987ae77185e068"
server_key_iv: "21092542bbb4d6802bc3c7ddc31796f2"
server_key_version: "encryptionkey@0"
I20260812 17:48:04.869553 14307 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:48:04.872287 14329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:04.873733 14307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:04.873905 14307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "ce4ceb9431b844d9a93d2e5af8e75d0f"
format_stamp: "Formatted at 2026-08-12 17:48:04 on dist-test-slave-sb2z"
server_key: "f43bd827b46b17c6ff987ae77185e068"
server_key_iv: "21092542bbb4d6802bc3c7ddc31796f2"
server_key_version: "encryptionkey@0"
I20260812 17:48:04.874034 14307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:04.887868 14307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:04.891430 14307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:04.891693 14307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:04.899130 14307 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:46423
I20260812 17:48:04.899155 14422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:46423 every 8 connection(s)
I20260812 17:48:04.900247 14307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:04.901214 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 14307
I20260812 17:48:04.901309 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:04.901593 13464 external_mini_cluster.cc:1428] Setting key de11f20d9e413decd5b250cd5bafca42
I20260812 17:48:04.903861 14423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:04.910177 14423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f: Bootstrap starting.
Aug 12 17:48:04 dist-test-slave-sb2z krb5kdc[14286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:04.912905 14423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:04.913928 14423 log.cc:824] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:04.916258 14423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f: No bootstrap required, opened a new log
I20260812 17:48:04.918344 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:04.903209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34110 (local address 127.13.38.62:46423)
0812 17:48:04.903791 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:04.903814 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:04.903863 (+ 49us) server_negotiation.cc:407] Connection header received
0812 17:48:04.904641 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:04.904660 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:04.905014 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:04.905378 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:04.906217 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:04.907276 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:04.908209 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:04.908868 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:04.909046 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:04.912291 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:04.912296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:04.912311 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:04.912343 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:04.914590 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:04.915094 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:04.915100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:04.915105 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:04.915224 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:04.915563 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:04.915569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:04.915571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:04.915994 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:04.916223 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:04.916567 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 17:48:04.916920 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":159,"threads_started":1}
I20260812 17:48:04.920612 14423 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4ceb9431b844d9a93d2e5af8e75d0f" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 46423 } }
I20260812 17:48:04.920974 14423 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:04.921061 14423 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce4ceb9431b844d9a93d2e5af8e75d0f, State: Initialized, Role: FOLLOWER
I20260812 17:48:04.921558 14423 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4ceb9431b844d9a93d2e5af8e75d0f" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 46423 } }
I20260812 17:48:04.921700 14423 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:04.921746 14423 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:04.921818 14423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:04.922806 14423 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4ceb9431b844d9a93d2e5af8e75d0f" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 46423 } }
I20260812 17:48:04.923166 14423 leader_election.cc:302] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce4ceb9431b844d9a93d2e5af8e75d0f; no voters:
I20260812 17:48:04.923568 14423 leader_election.cc:288] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:04.923710 14436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:04.923988 14436 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [term 1 LEADER]: Becoming Leader. State: Replica: ce4ceb9431b844d9a93d2e5af8e75d0f, State: Running, Role: LEADER
I20260812 17:48:04.924329 14436 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4ceb9431b844d9a93d2e5af8e75d0f" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 46423 } }
I20260812 17:48:04.925155 14423 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:04.926163 14440 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce4ceb9431b844d9a93d2e5af8e75d0f. Latest consensus state: current_term: 1 leader_uuid: "ce4ceb9431b844d9a93d2e5af8e75d0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4ceb9431b844d9a93d2e5af8e75d0f" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 46423 } } }
I20260812 17:48:04.926342 14440 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:04.926743 14449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:04.926717 14439 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce4ceb9431b844d9a93d2e5af8e75d0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4ceb9431b844d9a93d2e5af8e75d0f" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 46423 } } }
I20260812 17:48:04.926880 14439 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:04.929664 14449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:04.935995 14449 catalog_manager.cc:1347] Generated new cluster ID: 55af5df339ea4fcb8cd902e8b56e1473
I20260812 17:48:04.936094 14449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:04.954015 14449 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:04.955082 14449 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:04.965785 14449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce4ceb9431b844d9a93d2e5af8e75d0f: Generated new TSK 0
I20260812 17:48:04.966626 14449 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:48:05.005232 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:46423
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:05.134752 14471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:05.135032 14471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:05.135079 14471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:05.139185 14471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:05.139254 14471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:05.139317 14471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:05.145354 14471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:46423
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:05.146492 14471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:05.147590 14471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:05.158106 14482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:05.158135 14487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:05.158133 14485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:05.158424 14471 server_base.cc:1034] running on GCE node
I20260812 17:48:05.158890 14471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:05.159580 14471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:05.160796 14471 hybrid_clock.cc:648] HybridClock initialized: now 1786556885160758 us; error 51 us; skew 500 ppm
Aug 12 17:48:05 dist-test-slave-sb2z krb5kdc[14286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556885, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:05.164180 14471 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:48:05.165153 14471 webserver.cc:460] Webserver started at http://127.13.38.1:36619/ using document root <none> and password file <none>
I20260812 17:48:05.165748 14471 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:05.165848 14471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:05.166112 14471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:05.168300 14471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "2b19b7f177ad4cb1b858649c6c7287d1"
format_stamp: "Formatted at 2026-08-12 17:48:05 on dist-test-slave-sb2z"
server_key: "fad9256f450fc61aa0d7d4cc55eb2eac"
server_key_iv: "d094b500d09e3c25453a7c7388dd0413"
server_key_version: "encryptionkey@0"
I20260812 17:48:05.168919 14471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "2b19b7f177ad4cb1b858649c6c7287d1"
format_stamp: "Formatted at 2026-08-12 17:48:05 on dist-test-slave-sb2z"
server_key: "fad9256f450fc61aa0d7d4cc55eb2eac"
server_key_iv: "d094b500d09e3c25453a7c7388dd0413"
server_key_version: "encryptionkey@0"
I20260812 17:48:05.173224 14471 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:48:05.175900 14498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:05.177312 14471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:48:05.177454 14471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "2b19b7f177ad4cb1b858649c6c7287d1"
format_stamp: "Formatted at 2026-08-12 17:48:05 on dist-test-slave-sb2z"
server_key: "fad9256f450fc61aa0d7d4cc55eb2eac"
server_key_iv: "d094b500d09e3c25453a7c7388dd0413"
server_key_version: "encryptionkey@0"
I20260812 17:48:05.177595 14471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:05.200090 14471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:05.203976 14471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:05.204267 14471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:05.204980 14471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:05.205893 14471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:05.205974 14471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:05.206055 14471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:05.206100 14471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:05.216897 14471 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:36131
I20260812 17:48:05.216929 14684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:36131 every 8 connection(s)
I20260812 17:48:05.217850 14471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:05.223842 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 14471
I20260812 17:48:05.223979 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:05.224264 13464 external_mini_cluster.cc:1428] Setting key d0f30f456f25ec308afdfee67fc10486
Aug 12 17:48:05 dist-test-slave-sb2z krb5kdc[14286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556885, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:05.233012 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.219909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:53177 (local address 127.13.38.62:46423)
0812 17:48:05.220073 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:05.220077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:05.220967 (+ 890us) server_negotiation.cc:407] Connection header received
0812 17:48:05.221983 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:05.221994 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:05.222067 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:05.222184 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:05.223432 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.224135 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.225597 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.225847 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.225973 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.228837 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:05.228841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:05.228843 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:05.228867 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:05.230502 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.231209 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.231212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.231214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.231290 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.231727 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.231730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.231731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.231920 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:05.232004 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:05.232751 (+ 747us) server_negotiation.cc:299] Negotiation successful
0812 17:48:05.232868 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 17:48:05.234526 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.220287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:46423 (local address 127.13.38.1:53177)
0812 17:48:05.220783 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:05.220827 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:05.221756 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:05.222379 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:05.222386 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:05.222773 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:05.223268 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.223282 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.224281 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.224284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.225447 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.225458 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.226036 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.226041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.226274 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.226931 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:05.226949 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:05.228648 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:05.230640 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:05.230650 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:05.230673 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:05.231102 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:05.231392 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:05.231397 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:05.231400 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:05.231624 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:05.232054 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:05.232063 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:05.232451 (+ 388us) client_negotiation.cc:769] Sending connection context
0812 17:48:05.232711 (+ 260us) client_negotiation.cc:240] Negotiation successful
0812 17:48:05.233236 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":124,"threads_started":1}
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:48:05.235981 14686 heartbeater.cc:347] Connected to a master server at 127.13.38.62:46423
I20260812 17:48:05.236243 14686 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:05.236801 14686 heartbeater.cc:511] Master 127.13.38.62:46423 requested a full tablet report, sending...
I20260812 17:48:05.238476 14362 ts_manager.cc:194] Registered new tserver with Master: 2b19b7f177ad4cb1b858649c6c7287d1 (127.13.38.1:36131)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
I20260812 17:48:05.239861 14362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:53177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:48:05.256587 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:46423
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:05.389597 14692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:05.389878 14692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:05.389923 14692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:05.393965 14692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:05.394074 14692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:05.394174 14692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:05.399639 14692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:46423
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:05.400761 14692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:05.401832 14692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:05.411778 14706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:05.411808 14714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:05.411778 14707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:05.412184 14692 server_base.cc:1034] running on GCE node
I20260812 17:48:05.412606 14692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:05.413249 14692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:05.414417 14692 hybrid_clock.cc:648] HybridClock initialized: now 1786556885414381 us; error 55 us; skew 500 ppm
Aug 12 17:48:05 dist-test-slave-sb2z krb5kdc[14286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556885, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:05.417810 14692 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:48:05.418740 14692 webserver.cc:460] Webserver started at http://127.13.38.2:35749/ using document root <none> and password file <none>
I20260812 17:48:05.419284 14692 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:05.419344 14692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:05.419512 14692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:05.421680 14692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "70ca0b460d424325921e9612908025b7"
format_stamp: "Formatted at 2026-08-12 17:48:05 on dist-test-slave-sb2z"
server_key: "d256d313b45a76af33c17060c90e5b2f"
server_key_iv: "11854bce714df74b7e302f253d6c6a69"
server_key_version: "encryptionkey@0"
I20260812 17:48:05.422207 14692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "70ca0b460d424325921e9612908025b7"
format_stamp: "Formatted at 2026-08-12 17:48:05 on dist-test-slave-sb2z"
server_key: "d256d313b45a76af33c17060c90e5b2f"
server_key_iv: "11854bce714df74b7e302f253d6c6a69"
server_key_version: "encryptionkey@0"
I20260812 17:48:05.426072 14692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:48:05.428766 14723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:05.430037 14692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 17:48:05.430197 14692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "70ca0b460d424325921e9612908025b7"
format_stamp: "Formatted at 2026-08-12 17:48:05 on dist-test-slave-sb2z"
server_key: "d256d313b45a76af33c17060c90e5b2f"
server_key_iv: "11854bce714df74b7e302f253d6c6a69"
server_key_version: "encryptionkey@0"
I20260812 17:48:05.430311 14692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:05.446079 14692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:05.450172 14692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:05.450402 14692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:05.451133 14692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:05.452169 14692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:05.452244 14692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:05.452322 14692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:05.452373 14692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:05.463181 14692 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:39559
I20260812 17:48:05.463279 14907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:39559 every 8 connection(s)
I20260812 17:48:05.464303 14692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:05.464582 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 14692
I20260812 17:48:05.464684 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:05.465008 13464 external_mini_cluster.cc:1428] Setting key f87cf9399e705c8519eb5a4ae3247105
Aug 12 17:48:05 dist-test-slave-sb2z krb5kdc[14286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556885, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
I20260812 17:48:05.481870 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.466758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:43691 (local address 127.13.38.62:46423)
0812 17:48:05.466897 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:05.466902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:05.468221 (+ 1319us) server_negotiation.cc:407] Connection header received
0812 17:48:05.469727 (+ 1506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:05.469729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:05.469777 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:05.469851 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:05.471581 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.472486 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.473790 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.474238 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.474382 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.477147 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:05.477155 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:05.477160 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:05.477197 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:05.478876 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.479693 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.479696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.479698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.479788 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.480339 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.480346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.480358 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.480693 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:05.480816 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:05.481613 (+ 797us) server_negotiation.cc:299] Negotiation successful
0812 17:48:05.481739 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:48:05.483204 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.467188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:46423 (local address 127.13.38.2:43691)
0812 17:48:05.468056 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:05.468122 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:05.469457 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:05.470063 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:05.470075 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:05.470632 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:05.471382 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.471399 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.472639 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.472643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.473647 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.473659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.474421 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.474424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.474578 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.475255 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:05.475273 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:05.476934 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:05.479029 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:05.479035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:05.479051 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:05.479586 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:05.479988 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:05.479994 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:05.479999 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:05.480187 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:05.480861 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:05.480870 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:05.481289 (+ 419us) client_negotiation.cc:769] Sending connection context
0812 17:48:05.481531 (+ 242us) client_negotiation.cc:240] Negotiation successful
0812 17:48:05.481950 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":118,"threads_started":1}
I20260812 17:48:05.484609 14908 heartbeater.cc:347] Connected to a master server at 127.13.38.62:46423
I20260812 17:48:05.484918 14908 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:05.485615 14908 heartbeater.cc:511] Master 127.13.38.62:46423 requested a full tablet report, sending...
I20260812 17:48:05.486915 14362 ts_manager.cc:194] Registered new tserver with Master: 70ca0b460d424325921e9612908025b7 (127.13.38.2:39559)
I20260812 17:48:05.487591 14362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:43691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:48:05.504382 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:46423
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:05.635939 14925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:05.636241 14925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:05.636317 14925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:05.640542 14925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:05.640614 14925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:05.640700 14925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:05.646263 14925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:46423
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:05.647396 14925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:05.648519 14925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:05.658610 14935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:05.658653 14925 server_base.cc:1034] running on GCE node
W20260812 17:48:05.658614 14939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:05.658610 14936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:05.659250 14925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:05.659950 14925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:05.661145 14925 hybrid_clock.cc:648] HybridClock initialized: now 1786556885661134 us; error 34 us; skew 500 ppm
Aug 12 17:48:05 dist-test-slave-sb2z krb5kdc[14286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556885, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:05.669322 14925 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:48:05.670454 14925 webserver.cc:460] Webserver started at http://127.13.38.3:40279/ using document root <none> and password file <none>
I20260812 17:48:05.671150 14925 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:05.671265 14925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:05.671630 14925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:05.673975 14925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "c894c78d0d09438b9cafb9c4d0b8c839"
format_stamp: "Formatted at 2026-08-12 17:48:05 on dist-test-slave-sb2z"
server_key: "9723b57d908259156fe42ab6f4758b52"
server_key_iv: "f6171e63972386ea9514bbcf669319be"
server_key_version: "encryptionkey@0"
I20260812 17:48:05.674642 14925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "c894c78d0d09438b9cafb9c4d0b8c839"
format_stamp: "Formatted at 2026-08-12 17:48:05 on dist-test-slave-sb2z"
server_key: "9723b57d908259156fe42ab6f4758b52"
server_key_iv: "f6171e63972386ea9514bbcf669319be"
server_key_version: "encryptionkey@0"
I20260812 17:48:05.679093 14925 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 17:48:05.681972 14948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:05.683375 14925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:05.683619 14925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "c894c78d0d09438b9cafb9c4d0b8c839"
format_stamp: "Formatted at 2026-08-12 17:48:05 on dist-test-slave-sb2z"
server_key: "9723b57d908259156fe42ab6f4758b52"
server_key_iv: "f6171e63972386ea9514bbcf669319be"
server_key_version: "encryptionkey@0"
I20260812 17:48:05.683750 14925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:05.698191 14925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:05.701530 14925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:05.701743 14925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:05.702493 14925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:05.703414 14925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:05.703488 14925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:05.703581 14925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:05.703639 14925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:05.714231 14925 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:35247
I20260812 17:48:05.714251 15109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:35247 every 8 connection(s)
I20260812 17:48:05.715138 14925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:05.722910 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 14925
I20260812 17:48:05.723085 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:05.723387 13464 external_mini_cluster.cc:1428] Setting key bd099f57baa8733f45ce009cde5fa178
Aug 12 17:48:05 dist-test-slave-sb2z krb5kdc[14286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556885, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:05.729856 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.717060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:57195 (local address 127.13.38.62:46423)
0812 17:48:05.717199 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:05.717203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:05.718094 (+ 891us) server_negotiation.cc:407] Connection header received
0812 17:48:05.719038 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:05.719041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:05.719102 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:05.719183 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:05.720444 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.720998 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.722170 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.722399 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.722521 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.725649 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:05.725653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:05.725655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:05.725686 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:05.727423 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.728111 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.728114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.728116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.728210 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.728642 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.728645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.728647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.728870 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:05.728955 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:05.729599 (+ 644us) server_negotiation.cc:299] Negotiation successful
0812 17:48:05.729717 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 17:48:05.730803 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.717465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:46423 (local address 127.13.38.3:57195)
0812 17:48:05.717925 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:05.717967 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:05.718818 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:05.719356 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:05.719363 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:05.719805 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:05.720292 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.720306 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.721138 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.721141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.722025 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.722035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.722573 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.722578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.722766 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.723439 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:05.723457 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:05.725468 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:05.727612 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:05.727620 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:05.727639 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:05.727982 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:05.728322 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:05.728325 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:05.728327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:05.728510 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:05.729016 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:05.729023 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:05.729331 (+ 308us) client_negotiation.cc:769] Sending connection context
0812 17:48:05.729512 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 17:48:05.729825 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":114,"threads_started":1}
I20260812 17:48:05.732300 15110 heartbeater.cc:347] Connected to a master server at 127.13.38.62:46423
I20260812 17:48:05.732547 15110 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:05.733035 15110 heartbeater.cc:511] Master 127.13.38.62:46423 requested a full tablet report, sending...
I20260812 17:48:05.734196 14362 ts_manager.cc:194] Registered new tserver with Master: c894c78d0d09438b9cafb9c4d0b8c839 (127.13.38.3:35247)
I20260812 17:48:05.734802 14362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:57195
I20260812 17:48:05.737895 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:48:05.750020 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.741979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34126 (local address 127.13.38.62:46423)
0812 17:48:05.742213 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:05.742218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:05.742378 (+ 160us) server_negotiation.cc:407] Connection header received
0812 17:48:05.742548 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:05.742552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:05.742598 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:05.742693 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:05.743435 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.744062 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.744936 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.745201 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.745314 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.746714 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:05.746718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:05.746721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:05.746748 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:05.748139 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.748595 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.748599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.748601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.748688 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.748988 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.748991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.748993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.749303 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:05.749458 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:05.749712 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 17:48:05.749838 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 17:48:05.755944 14362 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:48:05.759392 14362 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:05.776939 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.770295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:39559 (local address 127.0.0.1:36916)
0812 17:48:05.771604 (+ 1309us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:05.771620 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:05.771708 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:05.772530 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:05.772532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:05.772549 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:05.772698 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.772708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.773756 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.773761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.775226 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.775238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.776281 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.776285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.776423 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.776468 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 17:48:05.776594 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 17:48:05.776723 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1212,"mutex_wait_us":71,"thread_start_us":138,"threads_started":1}
I20260812 17:48:05.776907 15141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.771065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36916 (local address 127.13.38.2:39559)
0812 17:48:05.771607 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:05.771674 (+ 67us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:05.771703 (+ 29us) server_negotiation.cc:407] Connection header received
0812 17:48:05.771880 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:05.771886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:05.772158 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:05.772338 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:05.772846 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.773613 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.775370 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.776138 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.776271 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.776492 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:05.776627 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 17:48:05.776723 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":173,"threads_started":1}
I20260812 17:48:05.777243 15131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.770234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:36131 (local address 127.0.0.1:43952)
0812 17:48:05.770938 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:05.770996 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:05.771142 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:05.772336 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:05.772343 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:05.772368 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:05.772555 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.772567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.774126 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.774129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.775278 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.775288 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.776668 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.776675 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.776864 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.776933 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 17:48:05.777010 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 17:48:05.777102 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":61,"thread_start_us":83,"threads_started":1}
I20260812 17:48:05.777674 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.771054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:35247 (local address 127.0.0.1:43252)
0812 17:48:05.771884 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:05.771902 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:05.772045 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:05.772628 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:05.772631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:05.772644 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:05.772747 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.772757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.774422 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.774425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.775466 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.775477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.776705 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.776708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.776862 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.776934 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 17:48:05.777472 (+ 538us) client_negotiation.cc:240] Negotiation successful
0812 17:48:05.777553 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":71,"thread_start_us":91,"threads_started":1}
I20260812 17:48:05.777820 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.770364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43952 (local address 127.13.38.1:36131)
0812 17:48:05.770758 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:05.770766 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:05.771303 (+ 537us) server_negotiation.cc:407] Connection header received
0812 17:48:05.771394 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:05.771400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:05.772183 (+ 783us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:05.772338 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:05.772707 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.773985 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.775417 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.776306 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.776420 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.776599 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:05.777556 (+ 957us) server_negotiation.cc:299] Negotiation successful
0812 17:48:05.777658 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":165,"threads_started":1}
I20260812 17:48:05.778127 15144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.771269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43252 (local address 127.13.38.3:35247)
0812 17:48:05.771776 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:05.771790 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:05.772153 (+ 363us) server_negotiation.cc:407] Connection header received
0812 17:48:05.772248 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:05.772253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:05.772472 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:05.772663 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:05.772845 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.774076 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.775635 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.776437 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.776535 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.776655 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:05.777530 (+ 875us) server_negotiation.cc:299] Negotiation successful
0812 17:48:05.777632 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":140,"threads_started":1}
I20260812 17:48:05.780707 14803 tablet_service.cc:1467] Processing CreateTablet for tablet 2ce587af4beb402fb749d4fdd789d853 (DEFAULT_TABLE table=test-table [id=ab68e6989d744a609c7305141d79264b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:05.780761 15018 tablet_service.cc:1467] Processing CreateTablet for tablet 2ce587af4beb402fb749d4fdd789d853 (DEFAULT_TABLE table=test-table [id=ab68e6989d744a609c7305141d79264b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:05.782071 15018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ce587af4beb402fb749d4fdd789d853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:05.782073 14803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ce587af4beb402fb749d4fdd789d853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:05.783250 14576 tablet_service.cc:1467] Processing CreateTablet for tablet 2ce587af4beb402fb749d4fdd789d853 (DEFAULT_TABLE table=test-table [id=ab68e6989d744a609c7305141d79264b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:05.784507 14576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ce587af4beb402fb749d4fdd789d853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:05.789880 15145 tablet_bootstrap.cc:492] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839: Bootstrap starting.
I20260812 17:48:05.791170 15146 tablet_bootstrap.cc:492] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7: Bootstrap starting.
I20260812 17:48:05.792192 15147 tablet_bootstrap.cc:492] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1: Bootstrap starting.
I20260812 17:48:05.792541 15145 tablet_bootstrap.cc:654] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:05.793632 15146 tablet_bootstrap.cc:654] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:05.794214 15145 log.cc:824] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:05.794837 15147 tablet_bootstrap.cc:654] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:05.794906 15146 log.cc:824] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:05.795993 15147 log.cc:824] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:05.796711 15145 tablet_bootstrap.cc:492] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839: No bootstrap required, opened a new log
I20260812 17:48:05.796972 15145 ts_tablet_manager.cc:1397] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 17:48:05.797048 15146 tablet_bootstrap.cc:492] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7: No bootstrap required, opened a new log
I20260812 17:48:05.797240 15146 ts_tablet_manager.cc:1397] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:48:05.798115 15147 tablet_bootstrap.cc:492] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1: No bootstrap required, opened a new log
I20260812 17:48:05.798322 15147 ts_tablet_manager.cc:1397] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:48:05.800684 15146 raft_consensus.cc:348] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b19b7f177ad4cb1b858649c6c7287d1" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 36131 } } peers { permanent_uuid: "70ca0b460d424325921e9612908025b7" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39559 } } peers { permanent_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 35247 } }
I20260812 17:48:05.801024 15146 raft_consensus.cc:374] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:05.801085 15146 raft_consensus.cc:729] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70ca0b460d424325921e9612908025b7, State: Initialized, Role: FOLLOWER
I20260812 17:48:05.801563 15146 consensus_queue.cc:260] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b19b7f177ad4cb1b858649c6c7287d1" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 36131 } } peers { permanent_uuid: "70ca0b460d424325921e9612908025b7" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39559 } } peers { permanent_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 35247 } }
I20260812 17:48:05.801816 15145 raft_consensus.cc:348] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b19b7f177ad4cb1b858649c6c7287d1" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 36131 } } peers { permanent_uuid: "70ca0b460d424325921e9612908025b7" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39559 } } peers { permanent_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 35247 } }
I20260812 17:48:05.802140 15145 raft_consensus.cc:374] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:05.802214 15145 raft_consensus.cc:729] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c894c78d0d09438b9cafb9c4d0b8c839, State: Initialized, Role: FOLLOWER
I20260812 17:48:05.802722 15146 ts_tablet_manager.cc:1428] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 17:48:05.802713 15145 consensus_queue.cc:260] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b19b7f177ad4cb1b858649c6c7287d1" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 36131 } } peers { permanent_uuid: "70ca0b460d424325921e9612908025b7" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39559 } } peers { permanent_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 35247 } }
I20260812 17:48:05.803200 14908 heartbeater.cc:503] Master 127.13.38.62:46423 was elected leader, sending a full tablet report...
I20260812 17:48:05.803859 15145 ts_tablet_manager.cc:1428] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 17:48:05.803965 15147 raft_consensus.cc:348] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b19b7f177ad4cb1b858649c6c7287d1" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 36131 } } peers { permanent_uuid: "70ca0b460d424325921e9612908025b7" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39559 } } peers { permanent_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 35247 } }
I20260812 17:48:05.804229 15147 raft_consensus.cc:374] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:05.804241 15110 heartbeater.cc:503] Master 127.13.38.62:46423 was elected leader, sending a full tablet report...
I20260812 17:48:05.804307 15147 raft_consensus.cc:729] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b19b7f177ad4cb1b858649c6c7287d1, State: Initialized, Role: FOLLOWER
I20260812 17:48:05.804759 15147 consensus_queue.cc:260] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b19b7f177ad4cb1b858649c6c7287d1" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 36131 } } peers { permanent_uuid: "70ca0b460d424325921e9612908025b7" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39559 } } peers { permanent_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 35247 } }
I20260812 17:48:05.805552 14686 heartbeater.cc:503] Master 127.13.38.62:46423 was elected leader, sending a full tablet report...
I20260812 17:48:05.805840 15147 ts_tablet_manager.cc:1428] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 17:48:05.882862 15153 raft_consensus.cc:482] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:05.883138 15153 raft_consensus.cc:504] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b19b7f177ad4cb1b858649c6c7287d1" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 36131 } } peers { permanent_uuid: "70ca0b460d424325921e9612908025b7" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39559 } } peers { permanent_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 35247 } }
I20260812 17:48:05.884402 15153 leader_election.cc:288] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b19b7f177ad4cb1b858649c6c7287d1 (127.13.38.1:36131), 70ca0b460d424325921e9612908025b7 (127.13.38.2:39559)
I20260812 17:48:05.889211 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.884893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:48703 (local address 127.13.38.1:36131)
0812 17:48:05.885038 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:05.885043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:05.885060 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:48:05.885210 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:05.885215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:05.885263 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:05.885350 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:05.885668 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.886454 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.887926 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.888579 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.888690 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.888761 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:05.889036 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 17:48:05.889105 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 17:48:05.889213 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.884753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:36131 (local address 127.13.38.3:48703)
0812 17:48:05.884894 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:05.884913 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:05.885074 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:05.885363 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:05.885365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:05.885384 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:05.885509 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.885515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.886571 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.886574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.887792 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.887802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.888736 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.888740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.888874 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.888906 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 17:48:05.888996 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 17:48:05.889067 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 17:48:05.889997 15141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.884876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:52111 (local address 127.13.38.2:39559)
0812 17:48:05.885049 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:05.885057 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:05.885557 (+ 500us) server_negotiation.cc:407] Connection header received
0812 17:48:05.885751 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:05.885753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:05.885794 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:05.885854 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:05.886315 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.887095 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.888725 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.889330 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.889446 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.889520 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:05.889806 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 17:48:05.889882 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:48:05.889998 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.884753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:39559 (local address 127.13.38.3:52111)
0812 17:48:05.885415 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:05.885428 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:05.885636 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:05.885897 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:05.885900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:05.885920 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:05.886169 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.886178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.887212 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.887214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.888541 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:05.888553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.889547 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.889550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:05.889656 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.889687 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 17:48:05.889784 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 17:48:05.889853 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":191,"thread_start_us":189,"threads_started":1}
I20260812 17:48:05.890097 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce587af4beb402fb749d4fdd789d853" candidate_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b19b7f177ad4cb1b858649c6c7287d1" is_pre_election: true
I20260812 17:48:05.890704 14616 raft_consensus.cc:2454] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c894c78d0d09438b9cafb9c4d0b8c839 in term 0.
I20260812 17:48:05.890790 14832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce587af4beb402fb749d4fdd789d853" candidate_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70ca0b460d424325921e9612908025b7" is_pre_election: true
I20260812 17:48:05.891103 14832 raft_consensus.cc:2454] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c894c78d0d09438b9cafb9c4d0b8c839 in term 0.
I20260812 17:48:05.891371 14956 leader_election.cc:302] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b19b7f177ad4cb1b858649c6c7287d1, c894c78d0d09438b9cafb9c4d0b8c839; no voters:
I20260812 17:48:05.891757 15153 raft_consensus.cc:2781] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:05.891845 15153 raft_consensus.cc:482] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:05.891877 15153 raft_consensus.cc:3037] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:05.893349 15153 raft_consensus.cc:504] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b19b7f177ad4cb1b858649c6c7287d1" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 36131 } } peers { permanent_uuid: "70ca0b460d424325921e9612908025b7" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39559 } } peers { permanent_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 35247 } }
I20260812 17:48:05.893780 15153 leader_election.cc:288] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [CANDIDATE]: Term 1 election: Requested vote from peers 2b19b7f177ad4cb1b858649c6c7287d1 (127.13.38.1:36131), 70ca0b460d424325921e9612908025b7 (127.13.38.2:39559)
I20260812 17:48:05.894197 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce587af4beb402fb749d4fdd789d853" candidate_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b19b7f177ad4cb1b858649c6c7287d1"
I20260812 17:48:05.894203 14832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce587af4beb402fb749d4fdd789d853" candidate_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70ca0b460d424325921e9612908025b7"
I20260812 17:48:05.894351 14616 raft_consensus.cc:3037] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:05.894363 14832 raft_consensus.cc:3037] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:05.895572 14832 raft_consensus.cc:2454] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c894c78d0d09438b9cafb9c4d0b8c839 in term 1.
I20260812 17:48:05.895776 14616 raft_consensus.cc:2454] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c894c78d0d09438b9cafb9c4d0b8c839 in term 1.
I20260812 17:48:05.895936 14950 leader_election.cc:302] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70ca0b460d424325921e9612908025b7, c894c78d0d09438b9cafb9c4d0b8c839; no voters:
I20260812 17:48:05.896144 15153 raft_consensus.cc:2781] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:05.896399 15153 raft_consensus.cc:686] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [term 1 LEADER]: Becoming Leader. State: Replica: c894c78d0d09438b9cafb9c4d0b8c839, State: Running, Role: LEADER
I20260812 17:48:05.896797 15153 consensus_queue.cc:237] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b19b7f177ad4cb1b858649c6c7287d1" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 36131 } } peers { permanent_uuid: "70ca0b460d424325921e9612908025b7" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39559 } } peers { permanent_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 35247 } }
I20260812 17:48:05.900790 14361 catalog_manager.cc:5495] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839 reported cstate change: term changed from 0 to 1, leader changed from <none> to c894c78d0d09438b9cafb9c4d0b8c839 (127.13.38.3). New cstate: current_term: 1 leader_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b19b7f177ad4cb1b858649c6c7287d1" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 36131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70ca0b460d424325921e9612908025b7" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c894c78d0d09438b9cafb9c4d0b8c839" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 35247 } health_report { overall_health: HEALTHY } } }
Aug 12 17:48:05 dist-test-slave-sb2z krb5kdc[14286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.1@KRBTEST.COM
I20260812 17:48:05.935169 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.925029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43956 (local address 127.13.38.1:36131)
0812 17:48:05.925213 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:05.925218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:05.925452 (+ 234us) server_negotiation.cc:407] Connection header received
0812 17:48:05.925579 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:05.925581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:05.925630 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:05.925704 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:05.926566 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.927224 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.928340 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.928574 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.928696 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.931349 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:05.931364 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:05.931382 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:05.931414 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:05.933187 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.933677 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.933684 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.933685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.933811 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.934125 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.934128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.934130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.934407 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:05.934646 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:05.934829 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 17:48:05.934962 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 17:48:05 dist-test-slave-sb2z krb5kdc[14286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556885, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 17:48:05 dist-test-slave-sb2z krb5kdc[14286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556885, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.38.1@KRBTEST.COM
W20260812 17:48:05.966168 14913 tablet.cc:2369] T 2ce587af4beb402fb749d4fdd789d853 P 70ca0b460d424325921e9612908025b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:48:05.966259 15111 tablet.cc:2369] T 2ce587af4beb402fb749d4fdd789d853 P c894c78d0d09438b9cafb9c4d0b8c839: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:48:05.967242 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.957904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43964 (local address 127.13.38.1:36131)
0812 17:48:05.958064 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:05.958068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:05.958273 (+ 205us) server_negotiation.cc:407] Connection header received
0812 17:48:05.958386 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:05.958388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:05.958436 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:05.958522 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:05.959277 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.959869 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.960807 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.961097 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.961196 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.963715 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:05.963719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:05.963721 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:05.963748 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:05.965301 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.965744 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.965747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.965749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.965859 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.966240 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.966243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.966245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.966513 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:05.966647 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:05.967005 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 17:48:05.967119 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260812 17:48:05.969359 14688 tablet.cc:2369] T 2ce587af4beb402fb749d4fdd789d853 P 2b19b7f177ad4cb1b858649c6c7287d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 17:48:05 dist-test-slave-sb2z krb5kdc[14286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 17:48:05 dist-test-slave-sb2z krb5kdc[14286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.1@KRBTEST.COM
I20260812 17:48:05.998180 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:05.988617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43974 (local address 127.13.38.1:36131)
0812 17:48:05.988816 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:05.988822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:05.989015 (+ 193us) server_negotiation.cc:407] Connection header received
0812 17:48:05.989128 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:05.989131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:05.989188 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:05.989258 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:05.990024 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.990563 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.991446 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:05.991743 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:05.991843 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:05.994709 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:05.994712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:05.994714 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:05.994744 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:05.996390 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.996824 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.996827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.996829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.996925 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:05.997271 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:05.997273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:05.997274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:05.997490 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:05.997609 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:05.997915 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 17:48:05.998050 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260812 17:48:06.000000 14540 tablet_service.cc:2908] Rejecting scan request for tablet 2ce587af4beb402fb749d4fdd789d853: Uninitialized: clean time has not yet been initialized
I20260812 17:48:06.003240 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 14471
I20260812 17:48:06.016736 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 14692
I20260812 17:48:06.031705 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 14925
I20260812 17:48:06.045338 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 14307
2026-08-12T17:48:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2777 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:06 dist-test-slave-sb2z krb5kdc[15184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:06 dist-test-slave-sb2z krb5kdc[15184](info): set up 2 sockets
Aug 12 17:48:06 dist-test-slave-sb2z krb5kdc[15184](info): commencing operation
krb5kdc: starting...
W20260812 17:48:07.319716 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:07 dist-test-slave-sb2z krb5kdc[15184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:48:07.463121 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:43541
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:43541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:07.583112 15204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:07.583465 15204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:07.583580 15204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:07.587672 15204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:07.587780 15204 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 17:48:07.587810 15204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:07.587855 15204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:07.587917 15204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:07.587954 15204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:07.593633 15204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:43541
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:43541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:07.594848 15204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:07.595979 15204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:07.605070 15217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:07.605093 15219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:07.605093 15221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:07.605242 15204 server_base.cc:1034] running on GCE node
I20260812 17:48:07.606160 15204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:07.607354 15204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:07.608544 15204 hybrid_clock.cc:648] HybridClock initialized: now 1786556887608512 us; error 43 us; skew 500 ppm
Aug 12 17:48:07 dist-test-slave-sb2z krb5kdc[15184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556887, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:07.611945 15204 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:48:07.613025 15204 webserver.cc:460] Webserver started at http://127.13.38.62:43335/ using document root <none> and password file <none>
I20260812 17:48:07.613615 15204 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:07.613679 15204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:07.613865 15204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:07.616204 15204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "bb04f48ec36c41e79222603bbd5a407e"
format_stamp: "Formatted at 2026-08-12 17:48:07 on dist-test-slave-sb2z"
server_key: "e22edfd5cc19798c8e0d2ed91ba85e24"
server_key_iv: "d123a52ca11272357de8f138147498c1"
server_key_version: "encryptionkey@0"
I20260812 17:48:07.616787 15204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "bb04f48ec36c41e79222603bbd5a407e"
format_stamp: "Formatted at 2026-08-12 17:48:07 on dist-test-slave-sb2z"
server_key: "e22edfd5cc19798c8e0d2ed91ba85e24"
server_key_iv: "d123a52ca11272357de8f138147498c1"
server_key_version: "encryptionkey@0"
I20260812 17:48:07.620960 15204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:48:07.623689 15232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:07.625181 15204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:07.625350 15204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "bb04f48ec36c41e79222603bbd5a407e"
format_stamp: "Formatted at 2026-08-12 17:48:07 on dist-test-slave-sb2z"
server_key: "e22edfd5cc19798c8e0d2ed91ba85e24"
server_key_iv: "d123a52ca11272357de8f138147498c1"
server_key_version: "encryptionkey@0"
I20260812 17:48:07.625483 15204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:07.652065 15204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:07.655776 15204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:07.656025 15204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:07.665261 15204 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:43541
I20260812 17:48:07.665292 15333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:43541 every 8 connection(s)
I20260812 17:48:07.666298 15204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:07.669269 15335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:07.671788 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 15204
I20260812 17:48:07.671924 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:07.672195 13464 external_mini_cluster.cc:1428] Setting key c804f5ffe63353a6a42704f33182740e
I20260812 17:48:07.676060 15335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e: Bootstrap starting.
I20260812 17:48:07.678476 15335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:07.679322 15335 log.cc:824] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:07.682353 15335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e: No bootstrap required, opened a new log
Aug 12 17:48:07 dist-test-slave-sb2z krb5kdc[15184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:07.687973 15335 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb04f48ec36c41e79222603bbd5a407e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43541 } }
I20260812 17:48:07.688398 15335 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:07.688463 15335 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb04f48ec36c41e79222603bbd5a407e, State: Initialized, Role: FOLLOWER
I20260812 17:48:07.689091 15335 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb04f48ec36c41e79222603bbd5a407e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43541 } }
I20260812 17:48:07.689299 15335 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:07.689376 15335 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:07.689498 15335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:07.689922 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:07.673809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37018 (local address 127.13.38.62:43541)
0812 17:48:07.674302 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:07.674319 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:07.674357 (+ 38us) server_negotiation.cc:407] Connection header received
0812 17:48:07.675116 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:07.675136 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:07.675521 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:07.675983 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:07.676958 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:07.678105 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:07.679188 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:07.679975 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:07.680187 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:07.683579 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:07.683588 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:07.683609 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:07.683650 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:07.686004 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:07.686559 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:07.686566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:07.686574 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:07.686750 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:07.687126 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:07.687134 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:07.687136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:07.687816 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:07.688041 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:07.688318 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 17:48:07.688696 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":146,"threads_started":1}
I20260812 17:48:07.690871 15335 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb04f48ec36c41e79222603bbd5a407e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43541 } }
I20260812 17:48:07.691298 15335 leader_election.cc:302] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb04f48ec36c41e79222603bbd5a407e; no voters:
I20260812 17:48:07.691740 15335 leader_election.cc:288] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:07.691852 15343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:07.692078 15343 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [term 1 LEADER]: Becoming Leader. State: Replica: bb04f48ec36c41e79222603bbd5a407e, State: Running, Role: LEADER
I20260812 17:48:07.692481 15343 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb04f48ec36c41e79222603bbd5a407e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43541 } }
I20260812 17:48:07.693300 15335 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:07.693650 15343 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb04f48ec36c41e79222603bbd5a407e. Latest consensus state: current_term: 1 leader_uuid: "bb04f48ec36c41e79222603bbd5a407e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb04f48ec36c41e79222603bbd5a407e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43541 } } }
I20260812 17:48:07.693811 15343 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:07.694288 15349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:07.694299 15344 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb04f48ec36c41e79222603bbd5a407e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb04f48ec36c41e79222603bbd5a407e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43541 } } }
I20260812 17:48:07.694571 15344 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:07.697645 15349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:07.703634 15349 catalog_manager.cc:1347] Generated new cluster ID: c67e4c770af944c58bb2346a26a3e52d
I20260812 17:48:07.703716 15349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:07.715487 15349 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:07.716504 15349 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:07.723430 15349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e: Generated new TSK 0
I20260812 17:48:07.724296 15349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:48:07.767410 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:43541
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:07.914960 15381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:07.915256 15381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:07.915303 15381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:07.919625 15381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:07.919698 15381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:07.919739 15381 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 17:48:07.919826 15381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:07.925458 15381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.38.62:43541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:07.926770 15381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:07.927904 15381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:07.940138 15390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:07.940194 15389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:07.940291 15381 server_base.cc:1034] running on GCE node
W20260812 17:48:07.940567 15394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:07.941058 15381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:07.941805 15381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:07.942971 15381 hybrid_clock.cc:648] HybridClock initialized: now 1786556887942952 us; error 38 us; skew 500 ppm
Aug 12 17:48:07 dist-test-slave-sb2z krb5kdc[15184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556887, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:07.946463 15381 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:48:07.947475 15381 webserver.cc:460] Webserver started at http://127.13.38.1:40741/ using document root <none> and password file <none>
I20260812 17:48:07.948143 15381 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:07.948244 15381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:07.948490 15381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:07.950873 15381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "8ce92640ffb5495eb9faf0fcb584b9a5"
format_stamp: "Formatted at 2026-08-12 17:48:07 on dist-test-slave-sb2z"
server_key: "c6df6dd7103743c278b71b3295fdd091"
server_key_iv: "867dcc964d47fdc2fda1803d27a681fb"
server_key_version: "encryptionkey@0"
I20260812 17:48:07.951611 15381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "8ce92640ffb5495eb9faf0fcb584b9a5"
format_stamp: "Formatted at 2026-08-12 17:48:07 on dist-test-slave-sb2z"
server_key: "c6df6dd7103743c278b71b3295fdd091"
server_key_iv: "867dcc964d47fdc2fda1803d27a681fb"
server_key_version: "encryptionkey@0"
I20260812 17:48:07.955860 15381 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:48:07.958892 15405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:07.960409 15381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:07.960587 15381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "8ce92640ffb5495eb9faf0fcb584b9a5"
format_stamp: "Formatted at 2026-08-12 17:48:07 on dist-test-slave-sb2z"
server_key: "c6df6dd7103743c278b71b3295fdd091"
server_key_iv: "867dcc964d47fdc2fda1803d27a681fb"
server_key_version: "encryptionkey@0"
I20260812 17:48:07.960721 15381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:07.978600 15381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:07.982841 15381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:07.983115 15381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:07.984925 15381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:07.985018 15381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:07.985098 15381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:07.985144 15381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 17:48:07 dist-test-slave-sb2z krb5kdc[15184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556887, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:07.999552 15381 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:39343
I20260812 17:48:07.999641 15598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:39343 every 8 connection(s)
I20260812 17:48:08.000977 15381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:08.002631 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:07.986007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:47823 (local address 127.13.38.62:43541)
0812 17:48:07.986172 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:07.986178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:07.987461 (+ 1283us) server_negotiation.cc:407] Connection header received
0812 17:48:07.989106 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:07.989113 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:07.989190 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:07.989276 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:07.991544 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:07.992243 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:07.993988 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:07.994283 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:07.994447 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:07.998001 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:07.998006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:07.998009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:07.998046 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:07.999940 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.000695 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.000699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.000701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.000793 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.001349 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.001352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.001354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.001655 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:08.001783 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.002375 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.002504 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 17:48:08.003669 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:07.986573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43541 (local address 127.13.38.1:47823)
0812 17:48:07.987191 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:07.987280 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:07.988781 (+ 1501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:07.989796 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:07.989808 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:07.990449 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:07.991291 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:07.991321 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:07.992420 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:07.992424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:07.993805 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:07.993819 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:07.994531 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:07.994537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:07.994799 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:07.995677 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:07.995710 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:07.997780 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:08.000164 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.000170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.000185 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.000567 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.000919 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.000926 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.000931 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.001164 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.001842 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:08.001848 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:08.002138 (+ 290us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.002308 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.002586 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":115,"threads_started":1}
I20260812 17:48:08.005185 15599 heartbeater.cc:347] Connected to a master server at 127.13.38.62:43541
I20260812 17:48:08.005364 15599 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:08.006063 15599 heartbeater.cc:511] Master 127.13.38.62:43541 requested a full tablet report, sending...
I20260812 17:48:08.007653 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 15381
I20260812 17:48:08.007774 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:08.007865 15257 ts_manager.cc:194] Registered new tserver with Master: 8ce92640ffb5495eb9faf0fcb584b9a5 (127.13.38.1:39343)
I20260812 17:48:08.008108 13464 external_mini_cluster.cc:1428] Setting key ecf547fd3a1d69e8529d3118bfd7fabb
I20260812 17:48:08.009476 15257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:47823
I20260812 17:48:08.016644 15608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.008033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43541 (local address 127.13.38.1:50867)
0812 17:48:08.008364 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.008380 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.008477 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.008979 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.008983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.009257 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:08.009388 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.009396 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.010402 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.010409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.011226 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.011236 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.011783 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.011787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.011885 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.012474 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:08.012496 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:08.012885 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:08.014898 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.014902 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.014904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.015156 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.015565 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.015568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.015570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.015667 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.016175 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:08.016178 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:08.016264 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.016347 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.016470 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":113,"threads_started":1}
I20260812 17:48:08.016846 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.008611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:50867 (local address 127.13.38.62:43541)
0812 17:48:08.008746 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.008751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.008763 (+ 12us) server_negotiation.cc:407] Connection header received
0812 17:48:08.008797 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.008800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.008847 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.008930 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:08.009540 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.010259 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.011365 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.011663 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.011775 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.013175 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:08.013180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:08.013183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:08.013209 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:08.014762 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.015291 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.015298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.015302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.015414 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.015770 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.015774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.015776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.016067 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:08.016189 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.016449 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.016615 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:48:08.043020 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:43541
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:08.173810 15619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:08.174175 15619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:08.174255 15619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:08.178520 15619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:08.178648 15619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:08.178730 15619 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 17:48:08.178815 15619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:08.184926 15619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.38.62:43541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:08.186264 15619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:08.187438 15619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:08.198364 15629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:08.198383 15625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:08.198412 15626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:08.198977 15619 server_base.cc:1034] running on GCE node
I20260812 17:48:08.199435 15619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:08.200188 15619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:08.201416 15619 hybrid_clock.cc:648] HybridClock initialized: now 1786556888201338 us; error 61 us; skew 500 ppm
Aug 12 17:48:08 dist-test-slave-sb2z krb5kdc[15184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556888, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:08.204849 15619 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:48:08.206143 15619 webserver.cc:460] Webserver started at http://127.13.38.2:36461/ using document root <none> and password file <none>
I20260812 17:48:08.206817 15619 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:08.206895 15619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:08.207157 15619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:08.209613 15619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "598b131da4264950884f87f0ab4f3d1b"
format_stamp: "Formatted at 2026-08-12 17:48:08 on dist-test-slave-sb2z"
server_key: "5361563433fb7e1f3f8b7984031a1e18"
server_key_iv: "bc31e6916ba322598bcb4f9f8adf1289"
server_key_version: "encryptionkey@0"
I20260812 17:48:08.210287 15619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "598b131da4264950884f87f0ab4f3d1b"
format_stamp: "Formatted at 2026-08-12 17:48:08 on dist-test-slave-sb2z"
server_key: "5361563433fb7e1f3f8b7984031a1e18"
server_key_iv: "bc31e6916ba322598bcb4f9f8adf1289"
server_key_version: "encryptionkey@0"
I20260812 17:48:08.215037 15619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 17:48:08.218240 15638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:08.219683 15619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:08.219828 15619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "598b131da4264950884f87f0ab4f3d1b"
format_stamp: "Formatted at 2026-08-12 17:48:08 on dist-test-slave-sb2z"
server_key: "5361563433fb7e1f3f8b7984031a1e18"
server_key_iv: "bc31e6916ba322598bcb4f9f8adf1289"
server_key_version: "encryptionkey@0"
I20260812 17:48:08.219957 15619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:08.237900 15619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:08.242141 15619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:08.242466 15619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:08.244429 15619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:08.244519 15619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:08.244603 15619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:08.244657 15619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 17:48:08 dist-test-slave-sb2z krb5kdc[15184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556888, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:08.258615 15619 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:33813
I20260812 17:48:08.258848 15838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:33813 every 8 connection(s)
I20260812 17:48:08.259891 15619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:08.261706 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 15619
I20260812 17:48:08.261833 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:08.262046 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.245461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:42309 (local address 127.13.38.62:43541)
0812 17:48:08.245616 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.245620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.246855 (+ 1235us) server_negotiation.cc:407] Connection header received
0812 17:48:08.248076 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.248080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.248144 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.248253 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:08.249763 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.250499 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.251997 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.252405 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.252546 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.257406 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:08.257412 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:08.257418 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:08.257455 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:08.259415 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.260077 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.260081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.260083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.260180 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.260651 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.260655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.260656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.260928 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:08.261023 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.261709 (+ 686us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.261850 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 17:48:08.262121 13464 external_mini_cluster.cc:1428] Setting key 794b7c1e19d1543515a153ae29303432
I20260812 17:48:08.263100 15658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.246058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43541 (local address 127.13.38.2:42309)
0812 17:48:08.246695 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.246740 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.247842 (+ 1102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.248455 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.248464 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.248982 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:08.249605 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.249622 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.250620 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.250623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.251848 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.251862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.252603 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.252609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.252959 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.253741 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:08.253762 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:08.257137 (+ 3375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:08.259633 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.259640 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.259653 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.259965 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.260283 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.260288 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.260290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.260524 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.261091 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:08.261097 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:08.261388 (+ 291us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.261644 (+ 256us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.262011 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":243,"threads_started":1}
I20260812 17:48:08.264599 15840 heartbeater.cc:347] Connected to a master server at 127.13.38.62:43541
I20260812 17:48:08.264767 15840 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:08.265445 15840 heartbeater.cc:511] Master 127.13.38.62:43541 requested a full tablet report, sending...
I20260812 17:48:08.266693 15259 ts_manager.cc:194] Registered new tserver with Master: 598b131da4264950884f87f0ab4f3d1b (127.13.38.2:33813)
I20260812 17:48:08.267390 15259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:42309
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:48:08.277347 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.267922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:43905 (local address 127.13.38.62:43541)
0812 17:48:08.268083 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.268090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.268347 (+ 257us) server_negotiation.cc:407] Connection header received
0812 17:48:08.268527 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.268530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.268581 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.268669 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:08.269325 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.270024 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.271402 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.271798 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.271888 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.273525 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:08.273529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:08.273531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:08.273570 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:08.275372 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.276006 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.276009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.276011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.276106 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.276413 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.276417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.276418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.276716 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:08.276816 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.277104 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.277223 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 17:48:08.277343 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.267956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43541 (local address 127.13.38.2:43905)
0812 17:48:08.268247 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.268270 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.268414 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.268689 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.268692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.268999 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:08.269178 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.269190 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.270181 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.270188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.271231 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.271245 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.271938 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.271944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.272089 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.272749 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:08.272770 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:08.273278 (+ 508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:08.275591 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.275597 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.275602 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.275887 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.276203 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.276206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.276207 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.276290 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.276807 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:08.276809 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:08.276912 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.277040 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.277181 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:48:08.297099 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:43541
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:08.432830 15852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:08.433142 15852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:08.433189 15852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:08.437393 15852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:08.437469 15852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:08.437510 15852 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 17:48:08.437544 15852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:08.443694 15852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.38.62:43541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:08.444921 15852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:08.446007 15852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:08.456920 15861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:08.456943 15865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:08.457178 15852 server_base.cc:1034] running on GCE node
W20260812 17:48:08.456931 15862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:08.457724 15852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:08.458714 15852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:08.459909 15852 hybrid_clock.cc:648] HybridClock initialized: now 1786556888459887 us; error 58 us; skew 500 ppm
Aug 12 17:48:08 dist-test-slave-sb2z krb5kdc[15184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556888, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:08.463176 15852 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:48:08.464440 15852 webserver.cc:460] Webserver started at http://127.13.38.3:34359/ using document root <none> and password file <none>
I20260812 17:48:08.465289 15852 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:08.465366 15852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:08.465651 15852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:08.468150 15852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "9ddee1048b5e45e7b7b445be2450bf44"
format_stamp: "Formatted at 2026-08-12 17:48:08 on dist-test-slave-sb2z"
server_key: "ebe712a40144394a7ff1010c82933c66"
server_key_iv: "7038945e8d1dbecd6c53811d3c0a820a"
server_key_version: "encryptionkey@0"
I20260812 17:48:08.468833 15852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "9ddee1048b5e45e7b7b445be2450bf44"
format_stamp: "Formatted at 2026-08-12 17:48:08 on dist-test-slave-sb2z"
server_key: "ebe712a40144394a7ff1010c82933c66"
server_key_iv: "7038945e8d1dbecd6c53811d3c0a820a"
server_key_version: "encryptionkey@0"
I20260812 17:48:08.473362 15852 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:48:08.476366 15879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:08.478055 15852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:08.478227 15852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "9ddee1048b5e45e7b7b445be2450bf44"
format_stamp: "Formatted at 2026-08-12 17:48:08 on dist-test-slave-sb2z"
server_key: "ebe712a40144394a7ff1010c82933c66"
server_key_iv: "7038945e8d1dbecd6c53811d3c0a820a"
server_key_version: "encryptionkey@0"
I20260812 17:48:08.478361 15852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:08.494881 15852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:08.498200 15852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:08.498486 15852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:08.500185 15852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:08.500275 15852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:08.500370 15852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:08.500415 15852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 17:48:08 dist-test-slave-sb2z krb5kdc[15184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556888, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:08.514222 15852 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:37841
I20260812 17:48:08.514542 16067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:37841 every 8 connection(s)
I20260812 17:48:08.515498 15852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:08.516242 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 15852
I20260812 17:48:08.516355 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:08.516672 13464 external_mini_cluster.cc:1428] Setting key c1cd388e2b6e136055db2b26a8b9164c
I20260812 17:48:08.518841 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.501160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:51907 (local address 127.13.38.62:43541)
0812 17:48:08.501327 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.501331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.503236 (+ 1905us) server_negotiation.cc:407] Connection header received
0812 17:48:08.505295 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.505299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.505364 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.505499 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:08.507442 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.508121 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.510212 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.510595 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.510726 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.514289 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:08.514293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:08.514297 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:08.514330 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:08.516198 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.516827 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.516830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.516833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.516930 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.517518 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.517521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.517522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.517791 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:08.517916 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.518506 (+ 590us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.518641 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 17:48:08.519826 15896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.502343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43541 (local address 127.13.38.3:51907)
0812 17:48:08.503061 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.503117 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.504900 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.505736 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.505743 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.506400 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:08.507255 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.507287 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.508275 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.508281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.510037 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.510054 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.510779 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.510782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.510941 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.511725 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:08.511755 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:08.514065 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:08.516327 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.516335 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.516353 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.516724 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.517219 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.517222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.517224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.517413 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.517913 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:08.517920 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:08.518262 (+ 342us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.518428 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.518781 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":176,"threads_started":1}
I20260812 17:48:08.521265 16069 heartbeater.cc:347] Connected to a master server at 127.13.38.62:43541
I20260812 17:48:08.521447 16069 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:08.521994 16069 heartbeater.cc:511] Master 127.13.38.62:43541 requested a full tablet report, sending...
I20260812 17:48:08.523150 15257 ts_manager.cc:194] Registered new tserver with Master: 9ddee1048b5e45e7b7b445be2450bf44 (127.13.38.3:37841)
I20260812 17:48:08.524039 15257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:51907
I20260812 17:48:08.531122 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:48:08.531142 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.523669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:50373 (local address 127.13.38.62:43541)
0812 17:48:08.523812 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.523818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.524221 (+ 403us) server_negotiation.cc:407] Connection header received
0812 17:48:08.524356 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.524362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.524416 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.524504 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:08.525060 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.525564 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.526385 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.526617 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.526727 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.527949 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:08.527952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:08.527955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:08.527985 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:08.529438 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.529874 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.529878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.529880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.530004 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.530299 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.530303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.530305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.530489 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:08.530602 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.530871 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.530996 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 17:48:08.531162 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.523568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43541 (local address 127.13.38.3:50373)
0812 17:48:08.524136 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.524152 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.524256 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.524535 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.524538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.524776 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:08.524880 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.524946 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.525700 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.525703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.526261 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.526269 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.526743 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.526746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.526848 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.527400 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:08.527412 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:08.527805 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:08.529559 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.529564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.529566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.529770 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.530109 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.530112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.530114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.530205 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.530591 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:08.530595 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:08.530701 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.530813 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.530966 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":201,"threads_started":1}
Aug 12 17:48:08 dist-test-slave-sb2z krb5kdc[15184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556888, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 17:48:08 dist-test-slave-sb2z krb5kdc[15184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556888, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:08.562768 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.553171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37028 (local address 127.13.38.62:43541)
0812 17:48:08.553339 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.553343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.553359 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:48:08.553406 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.553409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.553451 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.553538 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:08.554435 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.555016 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.555999 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.556224 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.556312 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.559285 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:08.559289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:08.559292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:08.559326 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:08.561022 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.561518 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.561523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.561524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.561617 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.561940 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.561942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.561943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.562172 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:08.562290 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.562517 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.562627 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 17:48:08.566594 15257 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:48:08.569952 15257 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:08.586079 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.579092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:33813 (local address 127.0.0.1:58558)
0812 17:48:08.579699 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.579755 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.579899 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.581114 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.581121 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.581144 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:08.581378 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.581386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.582914 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.582917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.584515 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.584528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.585471 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.585475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.585587 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.585626 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.585767 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.585908 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":158,"thread_start_us":110,"threads_started":1}
I20260812 17:48:08.586091 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.579896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58558 (local address 127.13.38.2:33813)
0812 17:48:08.580415 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.580438 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.580466 (+ 28us) server_negotiation.cc:407] Connection header received
0812 17:48:08.580565 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.580573 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.580801 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.581096 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:08.581514 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.582771 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.584674 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.585332 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.585470 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.585744 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.585819 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.585915 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":115,"threads_started":1}
I20260812 17:48:08.586320 16109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.579322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:37841 (local address 127.0.0.1:51638)
0812 17:48:08.580279 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.580293 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.580398 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.581195 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.581197 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.581209 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:08.581378 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.581386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.583077 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.583081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.584327 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.584337 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.585757 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.585760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.585959 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.585990 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.586077 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.586167 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":87,"thread_start_us":119,"threads_started":1}
I20260812 17:48:08.586396 16110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.580274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45674 (local address 127.13.38.1:39343)
0812 17:48:08.581196 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.581203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.581228 (+ 25us) server_negotiation.cc:407] Connection header received
0812 17:48:08.581306 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.581319 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.581495 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.581649 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:08.581874 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.582719 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.584664 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.585459 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.585556 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.585673 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.586162 (+ 489us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.586263 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":817,"thread_start_us":104,"threads_started":1}
I20260812 17:48:08.586723 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.579965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:39343 (local address 127.0.0.1:45674)
0812 17:48:08.580757 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.580770 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.580853 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.581637 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.581639 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.581656 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:08.581738 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.581743 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.582837 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.582843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.584518 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.584528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.585744 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.585748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.585991 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.586009 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.586074 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.586604 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":663,"thread_start_us":117,"threads_started":1}
I20260812 17:48:08.587199 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.579560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51638 (local address 127.13.38.3:37841)
0812 17:48:08.579983 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.579997 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.580506 (+ 509us) server_negotiation.cc:407] Connection header received
0812 17:48:08.580607 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.580614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.580838 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.581015 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:08.581809 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.582920 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.584464 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.585384 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.585516 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.585705 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.586944 (+ 1239us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.587051 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":94,"threads_started":1}
I20260812 17:48:08.589520 15955 tablet_service.cc:1467] Processing CreateTablet for tablet 46fb4e38558d495a98cecadeb5670586 (DEFAULT_TABLE table=test-table [id=2356ea6182b240d5825b42bfc0fb9561]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:08.589571 15727 tablet_service.cc:1467] Processing CreateTablet for tablet 46fb4e38558d495a98cecadeb5670586 (DEFAULT_TABLE table=test-table [id=2356ea6182b240d5825b42bfc0fb9561]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:08.589555 15489 tablet_service.cc:1467] Processing CreateTablet for tablet 46fb4e38558d495a98cecadeb5670586 (DEFAULT_TABLE table=test-table [id=2356ea6182b240d5825b42bfc0fb9561]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:08.590529 15955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46fb4e38558d495a98cecadeb5670586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:08.590638 15727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46fb4e38558d495a98cecadeb5670586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:08.590639 15489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46fb4e38558d495a98cecadeb5670586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:08.598291 16113 tablet_bootstrap.cc:492] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44: Bootstrap starting.
I20260812 17:48:08.599768 16115 tablet_bootstrap.cc:492] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b: Bootstrap starting.
I20260812 17:48:08.600785 16113 tablet_bootstrap.cc:654] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:08.601891 16113 log.cc:824] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:08.601984 16115 tablet_bootstrap.cc:654] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:08.602641 16114 tablet_bootstrap.cc:492] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5: Bootstrap starting.
I20260812 17:48:08.602881 16115 log.cc:824] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:08.604727 16114 tablet_bootstrap.cc:654] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:08.605089 16115 tablet_bootstrap.cc:492] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b: No bootstrap required, opened a new log
I20260812 17:48:08.605294 16113 tablet_bootstrap.cc:492] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44: No bootstrap required, opened a new log
I20260812 17:48:08.605309 16115 ts_tablet_manager.cc:1397] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:48:08.605481 16113 ts_tablet_manager.cc:1397] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 17:48:08.605693 16114 log.cc:824] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:08.607419 16114 tablet_bootstrap.cc:492] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5: No bootstrap required, opened a new log
I20260812 17:48:08.607648 16114 ts_tablet_manager.cc:1397] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 17:48:08.609882 16115 raft_consensus.cc:348] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } }
I20260812 17:48:08.609983 16113 raft_consensus.cc:348] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } }
I20260812 17:48:08.610201 16113 raft_consensus.cc:374] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:08.610201 16115 raft_consensus.cc:374] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:08.610265 16115 raft_consensus.cc:729] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 598b131da4264950884f87f0ab4f3d1b, State: Initialized, Role: FOLLOWER
I20260812 17:48:08.610298 16113 raft_consensus.cc:729] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ddee1048b5e45e7b7b445be2450bf44, State: Initialized, Role: FOLLOWER
I20260812 17:48:08.610805 16115 consensus_queue.cc:260] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } }
I20260812 17:48:08.610805 16113 consensus_queue.cc:260] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } }
I20260812 17:48:08.611742 15840 heartbeater.cc:503] Master 127.13.38.62:43541 was elected leader, sending a full tablet report...
I20260812 17:48:08.612030 16113 ts_tablet_manager.cc:1428] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 17:48:08.612452 16069 heartbeater.cc:503] Master 127.13.38.62:43541 was elected leader, sending a full tablet report...
I20260812 17:48:08.612627 16114 raft_consensus.cc:348] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } }
I20260812 17:48:08.612974 16114 raft_consensus.cc:374] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:08.613087 16114 raft_consensus.cc:729] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce92640ffb5495eb9faf0fcb584b9a5, State: Initialized, Role: FOLLOWER
I20260812 17:48:08.613277 16115 ts_tablet_manager.cc:1428] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260812 17:48:08.614121 16114 consensus_queue.cc:260] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } }
I20260812 17:48:08.614904 15599 heartbeater.cc:503] Master 127.13.38.62:43541 was elected leader, sending a full tablet report...
I20260812 17:48:08.615190 16114 ts_tablet_manager.cc:1428] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 17:48:08.747816 16120 raft_consensus.cc:482] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:08.748088 16120 raft_consensus.cc:504] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } }
I20260812 17:48:08.749383 16120 leader_election.cc:288] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ddee1048b5e45e7b7b445be2450bf44 (127.13.38.3:37841), 8ce92640ffb5495eb9faf0fcb584b9a5 (127.13.38.1:39343)
W20260812 17:48:08.752995 15600 tablet.cc:2369] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:48:08.754087 16110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.749839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:35663 (local address 127.13.38.1:39343)
0812 17:48:08.750041 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.750047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.750158 (+ 111us) server_negotiation.cc:407] Connection header received
0812 17:48:08.750334 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.750337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.750405 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.750497 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:08.750963 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.751727 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.752934 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.753497 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.753622 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.753692 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.753880 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.753959 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 17:48:08.754088 15658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.749784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:39343 (local address 127.13.38.2:35663)
0812 17:48:08.750049 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.750066 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.750188 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.750527 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.750533 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.750561 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:08.750721 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.750761 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.751848 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.751852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.752793 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.752804 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.753633 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.753637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.753739 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.753762 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.753863 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.753946 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260812 17:48:08.754840 15525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46fb4e38558d495a98cecadeb5670586" candidate_uuid: "598b131da4264950884f87f0ab4f3d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" is_pre_election: true
I20260812 17:48:08.755185 15525 raft_consensus.cc:2454] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 598b131da4264950884f87f0ab4f3d1b in term 0.
I20260812 17:48:08.755354 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.749852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:53019 (local address 127.13.38.3:37841)
0812 17:48:08.750042 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.750047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.750504 (+ 457us) server_negotiation.cc:407] Connection header received
0812 17:48:08.750637 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.750640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.750699 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.750787 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:08.751271 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.751985 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.754157 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.754701 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.754828 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.754899 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.755164 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.755238 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 17:48:08.755380 16124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.749785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:37841 (local address 127.13.38.2:53019)
0812 17:48:08.750352 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.750367 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.750526 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.750920 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.750923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.750947 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:08.751112 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.751139 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.752190 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.752192 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.754000 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.754011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.754872 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.754876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.755009 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.755054 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.755143 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.755219 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":65,"thread_start_us":112,"threads_started":1}
I20260812 17:48:08.755903 15640 leader_election.cc:302] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 598b131da4264950884f87f0ab4f3d1b, 8ce92640ffb5495eb9faf0fcb584b9a5; no voters:
I20260812 17:48:08.756131 15981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46fb4e38558d495a98cecadeb5670586" candidate_uuid: "598b131da4264950884f87f0ab4f3d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ddee1048b5e45e7b7b445be2450bf44" is_pre_election: true
I20260812 17:48:08.756230 16120 raft_consensus.cc:2781] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:08.756336 16120 raft_consensus.cc:482] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:08.756394 16120 raft_consensus.cc:3037] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:08.756466 15981 raft_consensus.cc:2454] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 598b131da4264950884f87f0ab4f3d1b in term 0.
I20260812 17:48:08.757658 16120 raft_consensus.cc:504] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } }
I20260812 17:48:08.758057 16120 leader_election.cc:288] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [CANDIDATE]: Term 1 election: Requested vote from peers 9ddee1048b5e45e7b7b445be2450bf44 (127.13.38.3:37841), 8ce92640ffb5495eb9faf0fcb584b9a5 (127.13.38.1:39343)
I20260812 17:48:08.758451 15981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46fb4e38558d495a98cecadeb5670586" candidate_uuid: "598b131da4264950884f87f0ab4f3d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ddee1048b5e45e7b7b445be2450bf44"
I20260812 17:48:08.758464 15525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46fb4e38558d495a98cecadeb5670586" candidate_uuid: "598b131da4264950884f87f0ab4f3d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5"
I20260812 17:48:08.758601 15981 raft_consensus.cc:3037] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:08.758615 15525 raft_consensus.cc:3037] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:08.760048 15525 raft_consensus.cc:2454] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 598b131da4264950884f87f0ab4f3d1b in term 1.
I20260812 17:48:08.760341 15981 raft_consensus.cc:2454] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 598b131da4264950884f87f0ab4f3d1b in term 1.
I20260812 17:48:08.760375 15640 leader_election.cc:302] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 598b131da4264950884f87f0ab4f3d1b, 8ce92640ffb5495eb9faf0fcb584b9a5; no voters:
I20260812 17:48:08.760566 16120 raft_consensus.cc:2781] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:08.760855 16120 raft_consensus.cc:686] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 1 LEADER]: Becoming Leader. State: Replica: 598b131da4264950884f87f0ab4f3d1b, State: Running, Role: LEADER
I20260812 17:48:08.761255 16120 consensus_queue.cc:237] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } }
W20260812 17:48:08.761688 15841 tablet.cc:2369] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:48:08.764979 15256 catalog_manager.cc:5495] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 598b131da4264950884f87f0ab4f3d1b (127.13.38.2). New cstate: current_term: 1 leader_uuid: "598b131da4264950884f87f0ab4f3d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } health_report { overall_health: UNKNOWN } } }
W20260812 17:48:08.767699 16071 tablet.cc:2369] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:48:08.785550 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.781210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58574 (local address 127.13.38.2:33813)
0812 17:48:08.781369 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.781373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.781412 (+ 39us) server_negotiation.cc:407] Connection header received
0812 17:48:08.781655 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.781657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.781716 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.781806 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:08.782137 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.782652 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.783733 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.783958 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.784055 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.784468 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:08.785001 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:08.785105 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.785263 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.785324 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 17:48:08.792364 15525 raft_consensus.cc:1261] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 1 FOLLOWER]: Refusing update from remote peer 598b131da4264950884f87f0ab4f3d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:08.792372 15981 raft_consensus.cc:1261] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44 [term 1 FOLLOWER]: Refusing update from remote peer 598b131da4264950884f87f0ab4f3d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:08.793228 16125 consensus_queue.cc:1035] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:08.795104 16125 consensus_queue.cc:1035] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:08.801438 16129 mvcc.cc:205] Tried to move back new op lower bound from 7317737016484540416 to 7317737016369803264. Current Snapshot: MvccSnapshot[applied={T|T < 7317737016484540416}]
I20260812 17:48:08.802136 16130 mvcc.cc:205] Tried to move back new op lower bound from 7317737016484540416 to 7317737016369803264. Current Snapshot: MvccSnapshot[applied={T|T < 7317737016484540416}]
I20260812 17:48:08.805831 16132 mvcc.cc:205] Tried to move back new op lower bound from 7317737016484540416 to 7317737016369803264. Current Snapshot: MvccSnapshot[applied={T|T < 7317737016484540416}]
I20260812 17:48:08.818326 15257 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:08.818617 15257 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:08.821648 15257 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e: Sending DeleteTablet for 3 replicas of tablet 46fb4e38558d495a98cecadeb5670586
I20260812 17:48:08.822831 15489 tablet_service.cc:1514] Processing DeleteTablet for tablet 46fb4e38558d495a98cecadeb5670586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:08 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:45674
I20260812 17:48:08.822945 15727 tablet_service.cc:1514] Processing DeleteTablet for tablet 46fb4e38558d495a98cecadeb5670586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:08 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:58558
I20260812 17:48:08.823758 15955 tablet_service.cc:1514] Processing DeleteTablet for tablet 46fb4e38558d495a98cecadeb5670586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:08 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:51638
I20260812 17:48:08.823946 16145 tablet_replica.cc:330] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5: stopping tablet replica
I20260812 17:48:08.823972 16143 tablet_replica.cc:330] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b: stopping tablet replica
I20260812 17:48:08.823931 15257 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 17:48:08.824362 16145 raft_consensus.cc:2229] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:08.824362 16143 raft_consensus.cc:2229] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 1 LEADER]: Raft consensus shutting down.
W20260812 17:48:08.824437 15257 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:08.824470 16147 tablet_replica.cc:330] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44: stopping tablet replica
I20260812 17:48:08.824797 16147 raft_consensus.cc:2229] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:08.824908 16143 raft_consensus.cc:2258] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:08.824911 16145 raft_consensus.cc:2258] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:08.825182 16147 raft_consensus.cc:2258] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:08.826795 16147 ts_tablet_manager.cc:1905] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:08.826797 16145 ts_tablet_manager.cc:1905] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:08.828327 16143 ts_tablet_manager.cc:1905] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:08.830225 16147 ts_tablet_manager.cc:1918] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:08.830309 16145 ts_tablet_manager.cc:1918] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:08.830392 16147 log.cc:1196] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/46fb4e38558d495a98cecadeb5670586
I20260812 17:48:08.830468 16145 log.cc:1196] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/wal/wals/46fb4e38558d495a98cecadeb5670586
I20260812 17:48:08.830816 16147 ts_tablet_manager.cc:1939] T 46fb4e38558d495a98cecadeb5670586 P 9ddee1048b5e45e7b7b445be2450bf44: Deleting consensus metadata
I20260812 17:48:08.830878 16145 ts_tablet_manager.cc:1939] T 46fb4e38558d495a98cecadeb5670586 P 8ce92640ffb5495eb9faf0fcb584b9a5: Deleting consensus metadata
I20260812 17:48:08.832152 15727 tablet_service.cc:1467] Processing CreateTablet for tablet 86890fed5697475cb13b553a249fb7eb (DEFAULT_TABLE table=test-table [id=16b90bad38854e37ae36a36bd9013088]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:08.832513 15727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86890fed5697475cb13b553a249fb7eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:08.832617 15241 catalog_manager.cc:4841] TS 9ddee1048b5e45e7b7b445be2450bf44 (127.13.38.3:37841): tablet 46fb4e38558d495a98cecadeb5670586 (table test-table [id=2356ea6182b240d5825b42bfc0fb9561]) successfully deleted
I20260812 17:48:08.832904 15236 catalog_manager.cc:4841] TS 8ce92640ffb5495eb9faf0fcb584b9a5 (127.13.38.1:39343): tablet 46fb4e38558d495a98cecadeb5670586 (table test-table [id=2356ea6182b240d5825b42bfc0fb9561]) successfully deleted
I20260812 17:48:08.833012 15955 tablet_service.cc:1467] Processing CreateTablet for tablet 86890fed5697475cb13b553a249fb7eb (DEFAULT_TABLE table=test-table [id=16b90bad38854e37ae36a36bd9013088]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:08.833272 15955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86890fed5697475cb13b553a249fb7eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:08.835613 16143 ts_tablet_manager.cc:1918] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:08.835810 16143 log.cc:1196] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/46fb4e38558d495a98cecadeb5670586
I20260812 17:48:08.836055 15489 tablet_service.cc:1467] Processing CreateTablet for tablet 86890fed5697475cb13b553a249fb7eb (DEFAULT_TABLE table=test-table [id=16b90bad38854e37ae36a36bd9013088]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:08.836193 16143 ts_tablet_manager.cc:1939] T 46fb4e38558d495a98cecadeb5670586 P 598b131da4264950884f87f0ab4f3d1b: Deleting consensus metadata
I20260812 17:48:08.836282 16113 tablet_bootstrap.cc:492] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44: Bootstrap starting.
I20260812 17:48:08.836371 15489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86890fed5697475cb13b553a249fb7eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:08.837733 16113 tablet_bootstrap.cc:654] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:08.838201 15235 catalog_manager.cc:4841] TS 598b131da4264950884f87f0ab4f3d1b (127.13.38.2:33813): tablet 46fb4e38558d495a98cecadeb5670586 (table test-table [id=2356ea6182b240d5825b42bfc0fb9561]) successfully deleted
I20260812 17:48:08.838739 16115 tablet_bootstrap.cc:492] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b: Bootstrap starting.
I20260812 17:48:08.839280 16114 tablet_bootstrap.cc:492] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5: Bootstrap starting.
I20260812 17:48:08.839838 16115 tablet_bootstrap.cc:654] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:08.840713 16114 tablet_bootstrap.cc:654] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:08.841219 16113 tablet_bootstrap.cc:492] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44: No bootstrap required, opened a new log
I20260812 17:48:08.841302 16113 ts_tablet_manager.cc:1397] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 17:48:08.841921 16113 raft_consensus.cc:348] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.842054 16113 raft_consensus.cc:374] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:08.842088 16113 raft_consensus.cc:729] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ddee1048b5e45e7b7b445be2450bf44, State: Initialized, Role: FOLLOWER
I20260812 17:48:08.842224 16114 tablet_bootstrap.cc:492] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5: No bootstrap required, opened a new log
I20260812 17:48:08.842216 16113 consensus_queue.cc:260] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.842329 16114 ts_tablet_manager.cc:1397] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:08.842468 16113 ts_tablet_manager.cc:1428] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 17:48:08.843290 16115 tablet_bootstrap.cc:492] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b: No bootstrap required, opened a new log
I20260812 17:48:08.843366 16115 ts_tablet_manager.cc:1397] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260812 17:48:08.843405 16114 raft_consensus.cc:348] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.843552 16114 raft_consensus.cc:374] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:08.843808 16114 raft_consensus.cc:729] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce92640ffb5495eb9faf0fcb584b9a5, State: Initialized, Role: FOLLOWER
I20260812 17:48:08.843987 16115 raft_consensus.cc:348] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.844118 16115 raft_consensus.cc:374] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:08.844157 16115 raft_consensus.cc:729] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 598b131da4264950884f87f0ab4f3d1b, State: Initialized, Role: FOLLOWER
I20260812 17:48:08.844079 16114 consensus_queue.cc:260] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.844275 16115 consensus_queue.cc:260] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.844529 16115 ts_tablet_manager.cc:1428] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 17:48:08.846119 16114 ts_tablet_manager.cc:1428] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 17:48:08.863613 16122 raft_consensus.cc:482] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:08.863862 16122 raft_consensus.cc:504] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.865301 16122 leader_election.cc:288] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 598b131da4264950884f87f0ab4f3d1b (127.13.38.2:33813), 9ddee1048b5e45e7b7b445be2450bf44 (127.13.38.3:37841)
I20260812 17:48:08.871145 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.866172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:50555 (local address 127.13.38.3:37841)
0812 17:48:08.866328 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.866335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.866352 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:48:08.866551 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.866555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.866606 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.866694 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:08.867035 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.867842 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.869797 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.870415 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.870534 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.870601 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.870929 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.871009 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:48:08.871150 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.865693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:44879 (local address 127.13.38.2:33813)
0812 17:48:08.865909 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.865914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.866387 (+ 473us) server_negotiation.cc:407] Connection header received
0812 17:48:08.866600 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.866604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.866731 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.866807 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:08.867268 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.867859 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.869810 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.870284 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.870389 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.870482 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.870926 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.871009 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:48:08.871662 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.865695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:37841 (local address 127.13.38.1:50555)
0812 17:48:08.866290 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.866302 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.866400 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.866738 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.866741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.866763 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:08.866907 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.866914 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.867972 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.867979 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.869644 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.869657 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.870669 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.870673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.870790 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.870815 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.871362 (+ 547us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.871446 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":119,"thread_start_us":117,"threads_started":1}
I20260812 17:48:08.871662 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.865595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:33813 (local address 127.13.38.1:44879)
0812 17:48:08.866120 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.866135 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.866447 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.866911 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.866914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.866945 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:08.867135 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.867142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.868009 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.868012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.869651 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.869657 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.870572 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.870576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.870760 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.870790 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.871311 (+ 521us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.871424 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":130,"thread_start_us":116,"threads_started":1}
I20260812 17:48:08.872383 15981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86890fed5697475cb13b553a249fb7eb" candidate_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ddee1048b5e45e7b7b445be2450bf44" is_pre_election: true
I20260812 17:48:08.872607 15981 raft_consensus.cc:2454] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ce92640ffb5495eb9faf0fcb584b9a5 in term 0.
I20260812 17:48:08.872576 15762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86890fed5697475cb13b553a249fb7eb" candidate_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "598b131da4264950884f87f0ab4f3d1b" is_pre_election: true
I20260812 17:48:08.872828 15762 raft_consensus.cc:2454] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ce92640ffb5495eb9faf0fcb584b9a5 in term 0.
I20260812 17:48:08.873025 15410 leader_election.cc:302] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ce92640ffb5495eb9faf0fcb584b9a5, 9ddee1048b5e45e7b7b445be2450bf44; no voters:
I20260812 17:48:08.873345 16122 raft_consensus.cc:2781] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:08.873404 16122 raft_consensus.cc:482] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:08.873430 16122 raft_consensus.cc:3037] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:08.874606 16122 raft_consensus.cc:504] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.875033 16122 leader_election.cc:288] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [CANDIDATE]: Term 1 election: Requested vote from peers 598b131da4264950884f87f0ab4f3d1b (127.13.38.2:33813), 9ddee1048b5e45e7b7b445be2450bf44 (127.13.38.3:37841)
I20260812 17:48:08.875402 15762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86890fed5697475cb13b553a249fb7eb" candidate_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "598b131da4264950884f87f0ab4f3d1b"
I20260812 17:48:08.875586 15762 raft_consensus.cc:3037] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:08.875614 15981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86890fed5697475cb13b553a249fb7eb" candidate_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ddee1048b5e45e7b7b445be2450bf44"
I20260812 17:48:08.875739 15981 raft_consensus.cc:3037] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:08.877061 15981 raft_consensus.cc:2454] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ce92640ffb5495eb9faf0fcb584b9a5 in term 1.
I20260812 17:48:08.877060 15762 raft_consensus.cc:2454] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ce92640ffb5495eb9faf0fcb584b9a5 in term 1.
I20260812 17:48:08.877408 15410 leader_election.cc:302] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ce92640ffb5495eb9faf0fcb584b9a5, 9ddee1048b5e45e7b7b445be2450bf44; no voters:
I20260812 17:48:08.877620 16122 raft_consensus.cc:2781] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:08.877964 16122 raft_consensus.cc:686] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ce92640ffb5495eb9faf0fcb584b9a5, State: Running, Role: LEADER
I20260812 17:48:08.878278 16122 consensus_queue.cc:237] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.881233 15259 catalog_manager.cc:5495] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ce92640ffb5495eb9faf0fcb584b9a5 (127.13.38.1). New cstate: current_term: 1 leader_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } health_report { overall_health: UNKNOWN } } }
Aug 12 17:48:08 dist-test-slave-sb2z krb5kdc[15184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556887, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:08.927438 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.917591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37030 (local address 127.13.38.62:43541)
0812 17:48:08.917758 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:08.917764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:08.918017 (+ 253us) server_negotiation.cc:407] Connection header received
0812 17:48:08.918141 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:08.918144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:08.918199 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:08.918308 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:08.919003 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.919792 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.920606 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.920922 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.921034 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.923804 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:08.923808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:08.923809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:08.923834 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:08.925488 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.926006 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.926010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.926012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.926098 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:08.926375 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:08.926378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:08.926379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:08.926595 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:08.926766 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:08.927091 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 17:48:08.927297 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":56}
I20260812 17:48:08.927439 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:08.917468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43541 (local address 127.0.0.1:37030)
0812 17:48:08.917908 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:08.917929 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:08.918045 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:08.918340 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:08.918342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:08.918766 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:08.918870 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.918877 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.919907 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.919910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.920472 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:08.920480 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:08.921041 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:08.921044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:08.921115 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:08.921762 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:08.921775 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:08.923628 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:08.925629 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.925640 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.925647 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.925895 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.926190 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:08.926195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:08.926196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:08.926280 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:08.926777 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:08.926787 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:08.926910 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 17:48:08.927029 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 17:48:08.927235 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":185,"threads_started":1}
I20260812 17:48:08.932663 15259 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:37030:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 17:48:08.933701 15259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:08.939620 15489 tablet_service.cc:1467] Processing CreateTablet for tablet b332360b5a8640429fe9352db1dae617 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7259d0820c3044a8beed97c135a8ecd0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 17:48:08.939821 15727 tablet_service.cc:1467] Processing CreateTablet for tablet b332360b5a8640429fe9352db1dae617 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7259d0820c3044a8beed97c135a8ecd0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 17:48:08.940086 15489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b332360b5a8640429fe9352db1dae617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:08.940220 15727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b332360b5a8640429fe9352db1dae617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:08.942809 16114 tablet_bootstrap.cc:492] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5: Bootstrap starting.
I20260812 17:48:08.943599 15955 tablet_service.cc:1467] Processing CreateTablet for tablet b332360b5a8640429fe9352db1dae617 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7259d0820c3044a8beed97c135a8ecd0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 17:48:08.943960 16114 tablet_bootstrap.cc:654] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:08.944037 15955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b332360b5a8640429fe9352db1dae617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:08.945392 16114 tablet_bootstrap.cc:492] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5: No bootstrap required, opened a new log
I20260812 17:48:08.945488 16114 ts_tablet_manager.cc:1397] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:08.946000 16114 raft_consensus.cc:348] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.946133 16114 raft_consensus.cc:374] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:08.946161 16115 tablet_bootstrap.cc:492] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: Bootstrap starting.
I20260812 17:48:08.946170 16114 raft_consensus.cc:729] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce92640ffb5495eb9faf0fcb584b9a5, State: Initialized, Role: FOLLOWER
I20260812 17:48:08.946419 16114 consensus_queue.cc:260] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.946800 16122 tablet_replica.cc:439] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } }
I20260812 17:48:08.946934 16122 tablet_replica.cc:442] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:08.947252 16113 tablet_bootstrap.cc:492] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44: Bootstrap starting.
I20260812 17:48:08.947309 16115 tablet_bootstrap.cc:654] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:08.948151 16114 ts_tablet_manager.cc:1428] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260812 17:48:08.948487 16113 tablet_bootstrap.cc:654] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:08.949909 16115 tablet_bootstrap.cc:492] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: No bootstrap required, opened a new log
I20260812 17:48:08.949944 16113 tablet_bootstrap.cc:492] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44: No bootstrap required, opened a new log
I20260812 17:48:08.950009 16115 ts_tablet_manager.cc:1397] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 17:48:08.950033 16113 ts_tablet_manager.cc:1397] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:08.950527 16113 raft_consensus.cc:348] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.950683 16113 raft_consensus.cc:374] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:08.950721 16113 raft_consensus.cc:729] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ddee1048b5e45e7b7b445be2450bf44, State: Initialized, Role: FOLLOWER
I20260812 17:48:08.950635 16115 raft_consensus.cc:348] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.950769 16115 raft_consensus.cc:374] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:08.950803 16115 raft_consensus.cc:729] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 598b131da4264950884f87f0ab4f3d1b, State: Initialized, Role: FOLLOWER
I20260812 17:48:08.950857 16113 consensus_queue.cc:260] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.950959 16115 consensus_queue.cc:260] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:08.951151 16113 ts_tablet_manager.cc:1428] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:08.951256 16115 ts_tablet_manager.cc:1428] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 17:48:08.951411 16121 tablet_replica.cc:439] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } }
I20260812 17:48:08.951625 16121 tablet_replica.cc:442] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:08.951826 16125 tablet_replica.cc:439] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } }
I20260812 17:48:08.951969 16125 tablet_replica.cc:442] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:09.252684 16125 raft_consensus.cc:482] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:09.252851 16125 raft_consensus.cc:504] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:09.253273 16125 leader_election.cc:288] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ce92640ffb5495eb9faf0fcb584b9a5 (127.13.38.1:39343), 9ddee1048b5e45e7b7b445be2450bf44 (127.13.38.3:37841)
I20260812 17:48:09.253847 15981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b332360b5a8640429fe9352db1dae617" candidate_uuid: "598b131da4264950884f87f0ab4f3d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ddee1048b5e45e7b7b445be2450bf44" is_pre_election: true
I20260812 17:48:09.253847 15525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b332360b5a8640429fe9352db1dae617" candidate_uuid: "598b131da4264950884f87f0ab4f3d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" is_pre_election: true
I20260812 17:48:09.254010 15981 raft_consensus.cc:2454] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 598b131da4264950884f87f0ab4f3d1b in term 0.
I20260812 17:48:09.254019 15525 raft_consensus.cc:2454] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 598b131da4264950884f87f0ab4f3d1b in term 0.
I20260812 17:48:09.254354 15640 leader_election.cc:302] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 598b131da4264950884f87f0ab4f3d1b, 8ce92640ffb5495eb9faf0fcb584b9a5; no voters:
I20260812 17:48:09.254563 16125 raft_consensus.cc:2781] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:09.254630 16125 raft_consensus.cc:482] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:09.254679 16125 raft_consensus.cc:3037] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:09.255988 16125 raft_consensus.cc:504] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:09.256397 16125 leader_election.cc:288] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [CANDIDATE]: Term 1 election: Requested vote from peers 8ce92640ffb5495eb9faf0fcb584b9a5 (127.13.38.1:39343), 9ddee1048b5e45e7b7b445be2450bf44 (127.13.38.3:37841)
I20260812 17:48:09.256801 15525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b332360b5a8640429fe9352db1dae617" candidate_uuid: "598b131da4264950884f87f0ab4f3d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5"
I20260812 17:48:09.256871 15981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b332360b5a8640429fe9352db1dae617" candidate_uuid: "598b131da4264950884f87f0ab4f3d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ddee1048b5e45e7b7b445be2450bf44"
I20260812 17:48:09.256946 15525 raft_consensus.cc:3037] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:09.257041 15981 raft_consensus.cc:3037] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:09.258167 15525 raft_consensus.cc:2454] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 598b131da4264950884f87f0ab4f3d1b in term 1.
I20260812 17:48:09.258167 15981 raft_consensus.cc:2454] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 598b131da4264950884f87f0ab4f3d1b in term 1.
I20260812 17:48:09.258579 15644 leader_election.cc:302] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 598b131da4264950884f87f0ab4f3d1b, 9ddee1048b5e45e7b7b445be2450bf44; no voters:
I20260812 17:48:09.258841 16125 raft_consensus.cc:2781] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:09.258996 16125 raft_consensus.cc:686] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [term 1 LEADER]: Becoming Leader. State: Replica: 598b131da4264950884f87f0ab4f3d1b, State: Running, Role: LEADER
I20260812 17:48:09.259151 16125 consensus_queue.cc:237] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } }
I20260812 17:48:09.259894 16120 tablet_replica.cc:439] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: TxnStatusTablet state changed. Reason: New leader 598b131da4264950884f87f0ab4f3d1b. Latest consensus state: current_term: 1 leader_uuid: "598b131da4264950884f87f0ab4f3d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } }
I20260812 17:48:09.260062 16120 tablet_replica.cc:442] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: This TxnStatusTablet replica's current role is: LEADER
I20260812 17:48:09.260408 16179 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 17:48:09.260637 16179 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 17:48:09.261082 15256 catalog_manager.cc:5495] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 598b131da4264950884f87f0ab4f3d1b (127.13.38.2). New cstate: current_term: 1 leader_uuid: "598b131da4264950884f87f0ab4f3d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } health_report { overall_health: UNKNOWN } } }
I20260812 17:48:09.326268 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:09.317924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:33813 (local address 127.0.0.1:58576)
0812 17:48:09.318385 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:09.318406 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:09.318544 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:09.322498 (+ 3954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:09.322502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:09.322518 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:09.322647 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:09.322665 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.323861 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.323880 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:09.324683 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:09.324696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.325365 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.325369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:09.325468 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:09.325531 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:09.325915 (+ 384us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:09.325935 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 17:48:09.326052 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 17:48:09.326138 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311}
I20260812 17:48:09.326287 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:09.318350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58576 (local address 127.13.38.2:33813)
0812 17:48:09.321088 (+ 2738us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:09.321094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:09.321113 (+ 19us) server_negotiation.cc:407] Connection header received
0812 17:48:09.321179 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:09.321182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:09.322187 (+ 1005us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:09.322299 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:09.322820 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.323695 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.324878 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.325224 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.325336 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:09.325681 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:09.325775 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:09.325851 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:09.326087 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 17:48:09.326145 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2601}
I20260812 17:48:09.329612 15981 raft_consensus.cc:1261] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44 [term 1 FOLLOWER]: Refusing update from remote peer 598b131da4264950884f87f0ab4f3d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:09.329648 15525 raft_consensus.cc:1261] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 1 FOLLOWER]: Refusing update from remote peer 598b131da4264950884f87f0ab4f3d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:09.330087 16125 consensus_queue.cc:1035] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:09.330247 16120 consensus_queue.cc:1035] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:09.331928 16122 tablet_replica.cc:439] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5: TxnStatusTablet state changed. Reason: New leader 598b131da4264950884f87f0ab4f3d1b. Latest consensus state: current_term: 1 leader_uuid: "598b131da4264950884f87f0ab4f3d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } }
I20260812 17:48:09.332060 16122 tablet_replica.cc:442] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:09.332319 16121 tablet_replica.cc:439] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44: TxnStatusTablet state changed. Reason: New leader 598b131da4264950884f87f0ab4f3d1b. Latest consensus state: current_term: 1 leader_uuid: "598b131da4264950884f87f0ab4f3d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } }
I20260812 17:48:09.332454 16121 tablet_replica.cc:442] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:09.333683 16120 tablet_replica.cc:439] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "598b131da4264950884f87f0ab4f3d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } }
I20260812 17:48:09.333794 16120 tablet_replica.cc:442] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: This TxnStatusTablet replica's current role is: LEADER
I20260812 17:48:09.335206 16122 tablet_replica.cc:439] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "598b131da4264950884f87f0ab4f3d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } }
I20260812 17:48:09.335322 16122 tablet_replica.cc:442] T b332360b5a8640429fe9352db1dae617 P 8ce92640ffb5495eb9faf0fcb584b9a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:09.336001 16125 tablet_replica.cc:439] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "598b131da4264950884f87f0ab4f3d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } }
I20260812 17:48:09.336128 16125 tablet_replica.cc:442] T b332360b5a8640429fe9352db1dae617 P 598b131da4264950884f87f0ab4f3d1b: This TxnStatusTablet replica's current role is: LEADER
I20260812 17:48:09.337468 16121 tablet_replica.cc:439] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "598b131da4264950884f87f0ab4f3d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce92640ffb5495eb9faf0fcb584b9a5" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39343 } } peers { permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 } } peers { permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 } } }
I20260812 17:48:09.337590 16121 tablet_replica.cc:442] T b332360b5a8640429fe9352db1dae617 P 9ddee1048b5e45e7b7b445be2450bf44: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:09.345870 16202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:09.341930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45682 (local address 127.13.38.1:39343)
0812 17:48:09.342270 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:09.342274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:09.342285 (+ 11us) server_negotiation.cc:407] Connection header received
0812 17:48:09.342352 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:09.342354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:09.342404 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:09.342542 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:09.342799 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.343355 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.344284 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.344551 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.344661 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:09.344982 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:09.345457 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:09.345588 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:09.345651 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 17:48:09.345708 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":68,"threads_started":1}
I20260812 17:48:09.358090 16206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:09.353820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:33813 (local address 127.13.38.1:50065)
0812 17:48:09.354291 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:09.354307 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:09.354431 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:09.354697 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:09.354699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:09.354704 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:09.354875 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:09.354880 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.355942 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.355945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:09.356632 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:09.356642 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.357279 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.357282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:09.357371 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:09.357437 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:09.357800 (+ 363us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:09.357808 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 17:48:09.357879 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 17:48:09.357954 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":207,"threads_started":1}
I20260812 17:48:09.358098 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:09.353865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:50065 (local address 127.13.38.2:33813)
0812 17:48:09.354054 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:09.354060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:09.354395 (+ 335us) server_negotiation.cc:407] Connection header received
0812 17:48:09.354546 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:09.354550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:09.354609 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:09.354696 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:09.354999 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.355816 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.356766 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.357077 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.357187 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:09.357583 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:09.357673 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:09.357749 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:09.357913 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 17:48:09.357967 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 17:48:09.366587 15981 raft_consensus.cc:1261] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44 [term 1 FOLLOWER]: Refusing update from remote peer 8ce92640ffb5495eb9faf0fcb584b9a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:09.366811 15762 raft_consensus.cc:1261] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b [term 1 FOLLOWER]: Refusing update from remote peer 8ce92640ffb5495eb9faf0fcb584b9a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:09.367086 16122 consensus_queue.cc:1035] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ddee1048b5e45e7b7b445be2450bf44" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:09.367663 16155 consensus_queue.cc:1035] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "598b131da4264950884f87f0ab4f3d1b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:09.394240 16216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:09.389840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:39343 (local address 127.13.38.2:49225)
0812 17:48:09.390241 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:09.390257 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:09.390375 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:09.390599 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:09.390601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:09.390610 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:09.390762 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:09.390769 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.391499 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.391504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:09.392541 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:09.392574 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.393368 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.393372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:09.393484 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:09.393565 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:09.393958 (+ 393us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:09.393963 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 17:48:09.394017 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 17:48:09.394091 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":132,"threads_started":1}
I20260812 17:48:09.394279 16202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:09.389981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:49225 (local address 127.13.38.1:39343)
0812 17:48:09.390117 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:09.390124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:09.390316 (+ 192us) server_negotiation.cc:407] Connection header received
0812 17:48:09.390460 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:09.390462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:09.390504 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:09.390567 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:09.390858 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.391373 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.392803 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.393179 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.393336 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:09.393707 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:09.393799 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:09.393878 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:09.394061 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 17:48:09.394129 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 17:48:09.424736 15257 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:09.424933 15257 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:09.427312 15257 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb04f48ec36c41e79222603bbd5a407e: Sending DeleteTablet for 3 replicas of tablet 86890fed5697475cb13b553a249fb7eb
I20260812 17:48:09.428020 15489 tablet_service.cc:1514] Processing DeleteTablet for tablet 86890fed5697475cb13b553a249fb7eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:09 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:45674
I20260812 17:48:09.428216 15955 tablet_service.cc:1514] Processing DeleteTablet for tablet 86890fed5697475cb13b553a249fb7eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:09 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:51638
I20260812 17:48:09.428527 16220 tablet_replica.cc:330] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5: stopping tablet replica
I20260812 17:48:09.428570 15727 tablet_service.cc:1514] Processing DeleteTablet for tablet 86890fed5697475cb13b553a249fb7eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:09 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:58558
I20260812 17:48:09.428643 16222 tablet_replica.cc:330] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44: stopping tablet replica
I20260812 17:48:09.428754 16220 raft_consensus.cc:2229] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:09.428800 16222 raft_consensus.cc:2229] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:09.429064 16222 raft_consensus.cc:2258] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:09.429273 16220 raft_consensus.cc:2258] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:09.429929 16220 ts_tablet_manager.cc:1905] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:09.429991 16222 ts_tablet_manager.cc:1905] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:09.430661 16223 tablet_replica.cc:330] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b: stopping tablet replica
I20260812 17:48:09.430828 16223 raft_consensus.cc:2229] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:09.430959 16223 raft_consensus.cc:2258] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:09.431720 16223 ts_tablet_manager.cc:1905] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:09.433445 16222 ts_tablet_manager.cc:1918] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:09.433563 16222 log.cc:1196] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/86890fed5697475cb13b553a249fb7eb
I20260812 17:48:09.433784 16220 ts_tablet_manager.cc:1918] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:09.433862 16222 ts_tablet_manager.cc:1939] T 86890fed5697475cb13b553a249fb7eb P 9ddee1048b5e45e7b7b445be2450bf44: Deleting consensus metadata
I20260812 17:48:09.433974 16220 log.cc:1196] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-0/wal/wals/86890fed5697475cb13b553a249fb7eb
I20260812 17:48:09.434329 16220 ts_tablet_manager.cc:1939] T 86890fed5697475cb13b553a249fb7eb P 8ce92640ffb5495eb9faf0fcb584b9a5: Deleting consensus metadata
I20260812 17:48:09.434798 15241 catalog_manager.cc:4841] TS 9ddee1048b5e45e7b7b445be2450bf44 (127.13.38.3:37841): tablet 86890fed5697475cb13b553a249fb7eb (table test-table [id=16b90bad38854e37ae36a36bd9013088]) successfully deleted
I20260812 17:48:09.434984 16223 ts_tablet_manager.cc:1918] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:09.435079 16223 log.cc:1196] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/86890fed5697475cb13b553a249fb7eb
I20260812 17:48:09.435190 15236 catalog_manager.cc:4841] TS 8ce92640ffb5495eb9faf0fcb584b9a5 (127.13.38.1:39343): tablet 86890fed5697475cb13b553a249fb7eb (table test-table [id=16b90bad38854e37ae36a36bd9013088]) successfully deleted
I20260812 17:48:09.435336 16223 ts_tablet_manager.cc:1939] T 86890fed5697475cb13b553a249fb7eb P 598b131da4264950884f87f0ab4f3d1b: Deleting consensus metadata
I20260812 17:48:09.436334 15235 catalog_manager.cc:4841] TS 598b131da4264950884f87f0ab4f3d1b (127.13.38.2:33813): tablet 86890fed5697475cb13b553a249fb7eb (table test-table [id=16b90bad38854e37ae36a36bd9013088]) successfully deleted
Aug 12 17:48:09 dist-test-slave-sb2z krb5kdc[15184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556888, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.38.1@KRBTEST.COM
I20260812 17:48:09.448170 16202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:09.435885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45694 (local address 127.13.38.1:39343)
0812 17:48:09.436002 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:09.436013 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:09.436493 (+ 480us) server_negotiation.cc:407] Connection header received
0812 17:48:09.436732 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:09.436735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:09.436782 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:09.436860 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:09.437791 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.438301 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.439623 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.439977 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.440054 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:09.444047 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:09.444062 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:09.444084 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:09.444116 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:09.446128 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:09.446569 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:09.446576 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:09.446578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:09.446726 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:09.446994 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:09.446998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:09.447000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:09.447373 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:09.447649 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:09.447839 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 17:48:09.447979 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 17:48:09.448838 15589 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45694
I20260812 17:48:09.460511 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:09.450896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37042 (local address 127.13.38.62:43541)
0812 17:48:09.451179 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:09.451186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:09.451201 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:48:09.451250 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:09.451253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:09.451292 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:09.451464 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:09.452369 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.453143 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.454343 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:09.454647 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:09.454752 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:09.456532 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:09.456535 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:09.456537 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:09.456577 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:09.458328 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:09.458918 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:09.458921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:09.458922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:09.459021 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:09.459323 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:09.459326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:09.459327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:09.459645 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:09.459761 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:09.460105 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 17:48:09.460241 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":74,"threads_started":1}
W20260812 17:48:09.461068 15257 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37042
I20260812 17:48:09.462711 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 15381
I20260812 17:48:09.473896 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 15619
I20260812 17:48:09.496361 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 15852
I20260812 17:48:09.513456 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 15204
2026-08-12T17:48:09Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3470 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:09 dist-test-slave-sb2z krb5kdc[16233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:09 dist-test-slave-sb2z krb5kdc[16233](info): set up 2 sockets
Aug 12 17:48:09 dist-test-slave-sb2z krb5kdc[16233](info): commencing operation
krb5kdc: starting...
W20260812 17:48:10.793421 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:10 dist-test-slave-sb2z krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:48:10.932032 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:43187
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:43187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:11.053709 16255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:11.054071 16255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:11.054150 16255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:11.058386 16255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:11.058483 16255 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 17:48:11.058516 16255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:11.058561 16255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:11.058596 16255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:11.058629 16255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:11.064287 16255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:43187
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:43187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:11.065534 16255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:11.066633 16255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:11.075778 16267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:11.075773 16263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:11.075785 16264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:11.076170 16255 server_base.cc:1034] running on GCE node
I20260812 17:48:11.076802 16255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:11.077996 16255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:11.079228 16255 hybrid_clock.cc:648] HybridClock initialized: now 1786556891079192 us; error 50 us; skew 500 ppm
Aug 12 17:48:11 dist-test-slave-sb2z krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556891, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:11.086071 16255 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:48:11.087174 16255 webserver.cc:460] Webserver started at http://127.13.38.62:42915/ using document root <none> and password file <none>
I20260812 17:48:11.087838 16255 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:11.087909 16255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:11.088100 16255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:11.090225 16255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "7938f86359c84b36938c86921e07e538"
format_stamp: "Formatted at 2026-08-12 17:48:11 on dist-test-slave-sb2z"
server_key: "8a72db75fe7a2ddbe264ac2e7b76c754"
server_key_iv: "712ec1b1920344acdba0de73f4baf07b"
server_key_version: "encryptionkey@0"
I20260812 17:48:11.090794 16255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "7938f86359c84b36938c86921e07e538"
format_stamp: "Formatted at 2026-08-12 17:48:11 on dist-test-slave-sb2z"
server_key: "8a72db75fe7a2ddbe264ac2e7b76c754"
server_key_iv: "712ec1b1920344acdba0de73f4baf07b"
server_key_version: "encryptionkey@0"
I20260812 17:48:11.094869 16255 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:48:11.097683 16280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:11.099119 16255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:11.099300 16255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "7938f86359c84b36938c86921e07e538"
format_stamp: "Formatted at 2026-08-12 17:48:11 on dist-test-slave-sb2z"
server_key: "8a72db75fe7a2ddbe264ac2e7b76c754"
server_key_iv: "712ec1b1920344acdba0de73f4baf07b"
server_key_version: "encryptionkey@0"
I20260812 17:48:11.099434 16255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:11.125183 16255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:11.128602 16255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:11.128841 16255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:11.137326 16255 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:43187
I20260812 17:48:11.137336 16376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:43187 every 8 connection(s)
I20260812 17:48:11.138381 16255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:11.140465 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 16255
I20260812 17:48:11.140646 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:11.140947 13464 external_mini_cluster.cc:1428] Setting key a058f15fd45007f1c84e8604515ced7e
I20260812 17:48:11.141889 16377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:11.148892 16377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538: Bootstrap starting.
Aug 12 17:48:11 dist-test-slave-sb2z krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:11.151729 16377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:11.152953 16377 log.cc:824] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:11.154994 16377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538: No bootstrap required, opened a new log
I20260812 17:48:11.156381 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.142435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47948 (local address 127.13.38.62:43187)
0812 17:48:11.142884 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:11.142901 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:11.142939 (+ 38us) server_negotiation.cc:407] Connection header received
0812 17:48:11.143685 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:11.143706 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:11.144057 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:11.144346 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:11.145324 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.146358 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.147683 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.148181 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.148310 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.151318 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:11.151323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:11.151339 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:11.151371 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:11.153397 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.153887 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.153893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.153898 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.154024 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.154333 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.154337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.154339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.154758 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:11.154970 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:11.155278 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 17:48:11.155638 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":150,"threads_started":1}
I20260812 17:48:11.158653 16377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7938f86359c84b36938c86921e07e538" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43187 } }
I20260812 17:48:11.158959 16377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:11.159047 16377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7938f86359c84b36938c86921e07e538, State: Initialized, Role: FOLLOWER
I20260812 17:48:11.159469 16377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7938f86359c84b36938c86921e07e538" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43187 } }
I20260812 17:48:11.159688 16377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:11.159780 16377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:11.159883 16377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:11.160831 16377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7938f86359c84b36938c86921e07e538" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43187 } }
I20260812 17:48:11.161206 16377 leader_election.cc:302] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7938f86359c84b36938c86921e07e538; no voters:
I20260812 17:48:11.161581 16377 leader_election.cc:288] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:11.161727 16382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:11.161994 16382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [term 1 LEADER]: Becoming Leader. State: Replica: 7938f86359c84b36938c86921e07e538, State: Running, Role: LEADER
I20260812 17:48:11.162312 16382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7938f86359c84b36938c86921e07e538" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43187 } }
I20260812 17:48:11.162855 16377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:11.164005 16384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7938f86359c84b36938c86921e07e538" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7938f86359c84b36938c86921e07e538" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43187 } } }
I20260812 17:48:11.164171 16384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:11.164453 16383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7938f86359c84b36938c86921e07e538. Latest consensus state: current_term: 1 leader_uuid: "7938f86359c84b36938c86921e07e538" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7938f86359c84b36938c86921e07e538" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43187 } } }
I20260812 17:48:11.164570 16383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:11.164705 16392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:11.168054 16392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:11.174456 16392 catalog_manager.cc:1347] Generated new cluster ID: f7a8315df0b34fd9ba2dd72702d37bba
I20260812 17:48:11.174585 16392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:11.182775 16392 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:11.183892 16392 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:11.188423 16392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538: Generated new TSK 0
I20260812 17:48:11.189275 16392 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:48:11.223265 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:43187
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:11.385306 16411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:11.385625 16411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:11.385685 16411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:11.390137 16411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:11.390221 16411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:11.390264 16411 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 17:48:11.390278 16411 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 17:48:11.390345 16411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:11.396306 16411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.38.62:43187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:11.397480 16411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:11.398573 16411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:11.410256 16419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:11.410521 16416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:11.410290 16417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:11.410786 16411 server_base.cc:1034] running on GCE node
I20260812 17:48:11.411258 16411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:11.412045 16411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:11.413255 16411 hybrid_clock.cc:648] HybridClock initialized: now 1786556891413239 us; error 41 us; skew 500 ppm
Aug 12 17:48:11 dist-test-slave-sb2z krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556891, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:11.416389 16411 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:48:11.417402 16411 webserver.cc:460] Webserver started at http://127.13.38.1:41537/ using document root <none> and password file <none>
I20260812 17:48:11.418066 16411 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:11.418169 16411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:11.418422 16411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:11.420792 16411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "b5e71aec20074411be80a4d2e616c8bb"
format_stamp: "Formatted at 2026-08-12 17:48:11 on dist-test-slave-sb2z"
server_key: "d74c982ba128b3f2c9ae98a8712a84d1"
server_key_iv: "35dca595c04932ce7a091666a3649b4e"
server_key_version: "encryptionkey@0"
I20260812 17:48:11.421468 16411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "b5e71aec20074411be80a4d2e616c8bb"
format_stamp: "Formatted at 2026-08-12 17:48:11 on dist-test-slave-sb2z"
server_key: "d74c982ba128b3f2c9ae98a8712a84d1"
server_key_iv: "35dca595c04932ce7a091666a3649b4e"
server_key_version: "encryptionkey@0"
I20260812 17:48:11.425593 16411 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:48:11.428246 16437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:11.429646 16411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:11.429776 16411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "b5e71aec20074411be80a4d2e616c8bb"
format_stamp: "Formatted at 2026-08-12 17:48:11 on dist-test-slave-sb2z"
server_key: "d74c982ba128b3f2c9ae98a8712a84d1"
server_key_iv: "35dca595c04932ce7a091666a3649b4e"
server_key_version: "encryptionkey@0"
I20260812 17:48:11.429888 16411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:11.445791 16411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:11.449141 16411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:11.449354 16411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:11.451263 16411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:11.451326 16411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:11.451399 16411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:11.451433 16411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 17:48:11 dist-test-slave-sb2z krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556891, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:11.469839 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.452725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:47377 (local address 127.13.38.62:43187)
0812 17:48:11.452864 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:11.452875 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:11.454431 (+ 1556us) server_negotiation.cc:407] Connection header received
0812 17:48:11.455809 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:11.455813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:11.455879 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:11.455989 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:11.457393 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.458442 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.460132 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.460371 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.460495 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.464990 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:11.464996 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:11.464999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:11.465035 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:11.467216 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.468073 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.468078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.468083 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.468179 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.468643 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.468646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.468648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.468899 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:11.469021 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:11.469579 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 17:48:11.469703 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 17:48:11.470849 16461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.453460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43187 (local address 127.13.38.1:47377)
0812 17:48:11.454243 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:11.454288 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:11.455503 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:11.456191 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:11.456198 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:11.456587 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:11.457157 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:11.457173 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.458610 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.458614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:11.459995 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:11.460007 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.460537 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.460542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:11.460740 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.461574 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:11.461591 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:11.464735 (+ 3144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:11.467364 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:11.467383 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:11.467412 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:11.467920 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:11.468276 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:11.468279 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:11.468281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:11.468486 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:11.469020 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:11.469027 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:11.469322 (+ 295us) client_negotiation.cc:769] Sending connection context
0812 17:48:11.469490 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 17:48:11.469769 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":305,"threads_started":1}
I20260812 17:48:11.474400 16411 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:45255
I20260812 17:48:11.474581 16624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:45255 every 8 connection(s)
I20260812 17:48:11.475497 16411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:11.477171 16629 heartbeater.cc:347] Connected to a master server at 127.13.38.62:43187
I20260812 17:48:11.477329 16629 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:11.477854 16629 heartbeater.cc:511] Master 127.13.38.62:43187 requested a full tablet report, sending...
I20260812 17:48:11.479422 16309 ts_manager.cc:194] Registered new tserver with Master: b5e71aec20074411be80a4d2e616c8bb (127.13.38.1:45255)
I20260812 17:48:11.480706 16309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:47377
I20260812 17:48:11.483168 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 16411
I20260812 17:48:11.483311 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:11.483690 13464 external_mini_cluster.cc:1428] Setting key fd66b2018b0299d8e384b2825b00aefb
I20260812 17:48:11.484527 16632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.476091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43187 (local address 127.13.38.1:39237)
0812 17:48:11.476430 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:11.476442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:11.476620 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:11.477089 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:11.477092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:11.477380 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:11.477542 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:11.477550 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.478476 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.478479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:11.479044 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:11.479052 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.479640 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.479644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:11.479741 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.480349 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:11.480361 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:11.480711 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:11.482778 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:11.482781 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:11.482783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:11.483017 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:11.483341 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:11.483346 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:11.483347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:11.483483 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:11.484041 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:11.484044 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:11.484154 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 17:48:11.484249 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 17:48:11.484362 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":156,"threads_started":1}
I20260812 17:48:11.484830 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.476095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:39237 (local address 127.13.38.62:43187)
0812 17:48:11.476238 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:11.476242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:11.476536 (+ 294us) server_negotiation.cc:407] Connection header received
0812 17:48:11.476756 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:11.476759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:11.476815 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:11.476859 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:11.477680 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.478315 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.479163 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.479488 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.479629 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.480871 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:11.480876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:11.480904 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:11.480927 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:11.482644 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.483133 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.483137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.483139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.483228 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.483647 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.483651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.483653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.483924 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:11.484027 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:11.484368 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 17:48:11.484524 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:48:11.515697 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:43187
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:11.640062 16635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:11.640364 16635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:11.640416 16635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:11.644690 16635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:11.644767 16635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:11.644810 16635 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 17:48:11.644846 16635 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 17:48:11.644915 16635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:11.650614 16635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.38.62:43187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:11.651870 16635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:11.652940 16635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:11.663153 16647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:11.663271 16641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:11.663614 16635 server_base.cc:1034] running on GCE node
W20260812 17:48:11.663254 16645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:11.664314 16635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:11.665069 16635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:11.666265 16635 hybrid_clock.cc:648] HybridClock initialized: now 1786556891666246 us; error 39 us; skew 500 ppm
Aug 12 17:48:11 dist-test-slave-sb2z krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556891, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:11.669399 16635 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:48:11.670512 16635 webserver.cc:460] Webserver started at http://127.13.38.2:41873/ using document root <none> and password file <none>
I20260812 17:48:11.671188 16635 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:11.671293 16635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:11.671602 16635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:11.673969 16635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "25665375389942e08e17cb942b777276"
format_stamp: "Formatted at 2026-08-12 17:48:11 on dist-test-slave-sb2z"
server_key: "da78d63b00d8992bc636d23ac77f2d92"
server_key_iv: "e8aa1e02008678317592d2b3097ad7fd"
server_key_version: "encryptionkey@0"
I20260812 17:48:11.674647 16635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "25665375389942e08e17cb942b777276"
format_stamp: "Formatted at 2026-08-12 17:48:11 on dist-test-slave-sb2z"
server_key: "da78d63b00d8992bc636d23ac77f2d92"
server_key_iv: "e8aa1e02008678317592d2b3097ad7fd"
server_key_version: "encryptionkey@0"
I20260812 17:48:11.678817 16635 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 17:48:11.681687 16661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:11.683255 16635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 17:48:11.683430 16635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "25665375389942e08e17cb942b777276"
format_stamp: "Formatted at 2026-08-12 17:48:11 on dist-test-slave-sb2z"
server_key: "da78d63b00d8992bc636d23ac77f2d92"
server_key_iv: "e8aa1e02008678317592d2b3097ad7fd"
server_key_version: "encryptionkey@0"
I20260812 17:48:11.683607 16635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:11.702838 16635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:11.706405 16635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:11.706629 16635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:11.708364 16635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:11.708442 16635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:11.708506 16635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:11.708557 16635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:11.718375 16635 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:41289
I20260812 17:48:11.718452 16860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:41289 every 8 connection(s)
I20260812 17:48:11.719611 16635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
Aug 12 17:48:11 dist-test-slave-sb2z krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556891, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:11.723819 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 16635
I20260812 17:48:11.723953 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:11.724227 13464 external_mini_cluster.cc:1428] Setting key f052fc112af2b301ec1cf810ed5507b8
I20260812 17:48:11.725941 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.709066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:56415 (local address 127.13.38.62:43187)
0812 17:48:11.709199 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:11.709204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:11.710999 (+ 1795us) server_negotiation.cc:407] Connection header received
0812 17:48:11.713021 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:11.713025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:11.713088 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:11.713230 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:11.715116 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.716045 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.718254 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.718587 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.718705 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.721800 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:11.721803 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:11.721806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:11.721834 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:11.723404 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.723999 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.724020 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.724025 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.724122 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.724552 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.724555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.724557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.724781 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:11.724882 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:11.725632 (+ 750us) server_negotiation.cc:299] Negotiation successful
0812 17:48:11.725767 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 17:48:11.726912 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.710331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43187 (local address 127.13.38.2:56415)
0812 17:48:11.710829 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:11.710881 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:11.712723 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:11.713503 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:11.713513 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:11.714135 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:11.714887 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:11.714922 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.716218 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.716221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:11.718091 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:11.718104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.718743 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.718746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:11.718899 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.719590 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:11.719616 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:11.721607 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:11.723564 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:11.723570 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:11.723586 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:11.723866 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:11.724219 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:11.724222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:11.724224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:11.724438 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:11.724967 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:11.724976 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:11.725312 (+ 336us) client_negotiation.cc:769] Sending connection context
0812 17:48:11.725531 (+ 219us) client_negotiation.cc:240] Negotiation successful
0812 17:48:11.725949 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":131,"threads_started":1}
I20260812 17:48:11.728358 16861 heartbeater.cc:347] Connected to a master server at 127.13.38.62:43187
I20260812 17:48:11.728564 16861 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:11.729115 16861 heartbeater.cc:511] Master 127.13.38.62:43187 requested a full tablet report, sending...
I20260812 17:48:11.730329 16309 ts_manager.cc:194] Registered new tserver with Master: 25665375389942e08e17cb942b777276 (127.13.38.2:41289)
I20260812 17:48:11.731074 16309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:56415
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
I20260812 17:48:11.740455 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.731263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:54049 (local address 127.13.38.62:43187)
0812 17:48:11.731411 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:11.731419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:11.731766 (+ 347us) server_negotiation.cc:407] Connection header received
0812 17:48:11.731997 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:11.732002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:11.732065 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:11.732124 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:11.732878 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.733578 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.735010 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.735264 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.735353 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.736766 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:11.736775 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:11.736777 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:11.736801 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:11.738508 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.738996 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.738998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.739000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.739116 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.739439 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.739442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.739443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.739741 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:11.739898 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:11.740213 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 17:48:11.740329 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 17:48:11.740644 16871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.731156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43187 (local address 127.13.38.2:54049)
0812 17:48:11.731673 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:11.731692 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:11.731850 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:11.732295 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:11.732301 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:11.732559 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:11.732724 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:11.732736 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.733727 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.733733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:11.734845 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:11.734859 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.735406 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.735412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:11.735561 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.736099 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:11.736119 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:11.736616 (+ 497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:11.738633 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:11.738636 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:11.738638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:11.738845 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:11.739259 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:11.739261 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:11.739263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:11.739347 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:11.739851 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:11.739854 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:11.739983 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 17:48:11.740129 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 17:48:11.740326 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:48:11.759910 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:43187
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:11.879503 16875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:11.879859 16875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:11.879909 16875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:11.884212 16875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:11.884289 16875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:11.884332 16875 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 17:48:11.884348 16875 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 17:48:11.884400 16875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:11.890215 16875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.38.62:43187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:11.891399 16875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:11.892493 16875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:11.902527 16893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:11.902506 16892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:11.902489 16897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:11.903023 16875 server_base.cc:1034] running on GCE node
I20260812 17:48:11.903450 16875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:11.904340 16875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:11.905556 16875 hybrid_clock.cc:648] HybridClock initialized: now 1786556891905495 us; error 46 us; skew 500 ppm
Aug 12 17:48:11 dist-test-slave-sb2z krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556891, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:11.908825 16875 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:48:11.909773 16875 webserver.cc:460] Webserver started at http://127.13.38.3:33609/ using document root <none> and password file <none>
I20260812 17:48:11.910377 16875 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:11.910444 16875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:11.910630 16875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:11.912932 16875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "27a2a8f42e3a495bab1692890e3c7516"
format_stamp: "Formatted at 2026-08-12 17:48:11 on dist-test-slave-sb2z"
server_key: "f5008294bff165342c62ac186ad2d5d3"
server_key_iv: "46eec55ff41c323c68f988a03443e46c"
server_key_version: "encryptionkey@0"
I20260812 17:48:11.913527 16875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "27a2a8f42e3a495bab1692890e3c7516"
format_stamp: "Formatted at 2026-08-12 17:48:11 on dist-test-slave-sb2z"
server_key: "f5008294bff165342c62ac186ad2d5d3"
server_key_iv: "46eec55ff41c323c68f988a03443e46c"
server_key_version: "encryptionkey@0"
I20260812 17:48:11.917646 16875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:48:11.920336 16909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:11.921824 16875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:11.921973 16875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "27a2a8f42e3a495bab1692890e3c7516"
format_stamp: "Formatted at 2026-08-12 17:48:11 on dist-test-slave-sb2z"
server_key: "f5008294bff165342c62ac186ad2d5d3"
server_key_iv: "46eec55ff41c323c68f988a03443e46c"
server_key_version: "encryptionkey@0"
I20260812 17:48:11.922099 16875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:11.941754 16875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:11.945186 16875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:11.945417 16875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:11.947022 16875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:11.947108 16875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:11.947187 16875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:11.947240 16875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 17:48:11 dist-test-slave-sb2z krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556891, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:11.959391 16875 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:44381
I20260812 17:48:11.960512 17094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:44381 every 8 connection(s)
I20260812 17:48:11.960772 16875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:11.962620 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.948176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:33475 (local address 127.13.38.62:43187)
0812 17:48:11.948341 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:11.948348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:11.949741 (+ 1393us) server_negotiation.cc:407] Connection header received
0812 17:48:11.951011 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:11.951015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:11.951088 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:11.951195 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:11.952513 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.953199 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.954465 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.954703 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.954822 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.958292 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:11.958296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:11.958298 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:11.958327 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:11.960250 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.960996 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.961000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.961002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.961098 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.961527 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.961531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.961533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.961806 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:11.961903 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:11.962368 (+ 465us) server_negotiation.cc:299] Negotiation successful
0812 17:48:11.962481 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 17:48:11.963629 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.948964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43187 (local address 127.13.38.3:33475)
0812 17:48:11.949558 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:11.949605 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:11.950779 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:11.951394 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:11.951403 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:11.951848 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:11.952353 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:11.952366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.953330 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.953335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:11.954329 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:11.954339 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.954888 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.954893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:11.955122 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.955878 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:11.955909 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:11.958077 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:11.960433 (+ 2356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:11.960439 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:11.960452 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:11.960859 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:11.961212 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:11.961215 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:11.961217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:11.961401 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:11.961901 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:11.961906 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:11.962146 (+ 240us) client_negotiation.cc:769] Sending connection context
0812 17:48:11.962312 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 17:48:11.962581 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":139,"threads_started":1}
I20260812 17:48:11.965094 17095 heartbeater.cc:347] Connected to a master server at 127.13.38.62:43187
I20260812 17:48:11.965279 17095 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:11.966208 17095 heartbeater.cc:511] Master 127.13.38.62:43187 requested a full tablet report, sending...
I20260812 17:48:11.967360 16309 ts_manager.cc:194] Registered new tserver with Master: 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3:44381)
I20260812 17:48:11.968189 16309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:33475
I20260812 17:48:11.969049 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 16875
I20260812 17:48:11.969180 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:11.969498 13464 external_mini_cluster.cc:1428] Setting key df2aa8be95db4f1e0648863240f8fff9
I20260812 17:48:11.971663 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:48:11.976040 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.967644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:60055 (local address 127.13.38.62:43187)
0812 17:48:11.967769 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:11.967773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:11.968013 (+ 240us) server_negotiation.cc:407] Connection header received
0812 17:48:11.968120 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:11.968123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:11.968163 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:11.968231 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:11.968763 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.969458 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.970654 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.970902 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.971014 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.972297 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:11.972301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:11.972304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:11.972359 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:11.974178 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.974775 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.974778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.974779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.974857 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:11.975177 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:11.975180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:11.975181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:11.975352 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:11.975430 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:11.975789 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 17:48:11.975887 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 17:48:11.976166 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.967596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43187 (local address 127.13.38.3:60055)
0812 17:48:11.967911 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:11.967928 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:11.968036 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:11.968288 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:11.968291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:11.968537 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:11.968645 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:11.968652 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.969658 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.969661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:11.970533 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:11.970544 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.971030 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.971035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:11.971134 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:11.971769 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:11.971788 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:11.972152 (+ 364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:11.974307 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:11.974315 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:11.974319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:11.974666 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:11.974954 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:11.974958 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:11.974963 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:11.975078 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:11.975450 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:11.975455 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:11.975602 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 17:48:11.975735 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 17:48:11.975913 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":2,"thread_start_us":100,"threads_started":1}
Aug 12 17:48:11 dist-test-slave-sb2z krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 17:48:11 dist-test-slave-sb2z krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:12.003203 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:11.994084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47954 (local address 127.13.38.62:43187)
0812 17:48:11.994253 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:11.994257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:11.994273 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:48:11.994322 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:11.994324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:11.994367 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:11.994462 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:11.995315 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.995983 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.997025 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:11.997253 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:11.997352 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.000108 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:12.000111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:12.000113 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:12.000145 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:12.001576 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:12.002019 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:12.002022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:12.002023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:12.002114 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:12.002378 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:12.002380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:12.002381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:12.002574 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:12.002684 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.002892 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.002994 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 17:48:12.006861 16309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:48:12.010651 16309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:12.028906 17121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.020743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:41289 (local address 127.0.0.1:33344)
0812 17:48:12.021886 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:12.021938 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:12.022071 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:12.022810 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:12.022816 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:12.022841 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:12.023028 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.023035 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.024458 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.024465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.027211 (+ 2746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.027223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.028420 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.028424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.028524 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.028552 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 17:48:12.028649 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 17:48:12.028730 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"mutex_wait_us":1332,"thread_start_us":154,"threads_started":1}
I20260812 17:48:12.028975 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.021091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33344 (local address 127.13.38.2:41289)
0812 17:48:12.022314 (+ 1223us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.022329 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.022357 (+ 28us) server_negotiation.cc:407] Connection header received
0812 17:48:12.022458 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.022463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.022638 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.022801 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:12.023161 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.024303 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.027371 (+ 3068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.028168 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.028277 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.028446 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.028704 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.028813 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1098,"thread_start_us":119,"threads_started":1}
I20260812 17:48:12.029543 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.021843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39192 (local address 127.13.38.1:45255)
0812 17:48:12.022374 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.022381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.023305 (+ 924us) server_negotiation.cc:407] Connection header received
0812 17:48:12.023393 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.023398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.023656 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.023799 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:12.024222 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.025455 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.027647 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.028565 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.028676 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.028819 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.029281 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.029394 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":113,"threads_started":1}
I20260812 17:48:12.029841 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.021746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52898 (local address 127.13.38.3:44381)
0812 17:48:12.022434 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.022443 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.022463 (+ 20us) server_negotiation.cc:407] Connection header received
0812 17:48:12.022526 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.022529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.022671 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.022843 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:12.023216 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.024494 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.026738 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.027909 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.028044 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.028324 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.029575 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.029693 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":135,"threads_started":1}
I20260812 17:48:12.029469 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.020740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:44381 (local address 127.0.0.1:52898)
0812 17:48:12.021858 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:12.021924 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:12.022072 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:12.022851 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:12.022855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:12.022877 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:12.023036 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.023044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.024707 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.024710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.026561 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.026574 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.028070 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.028076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.028215 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.028279 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 17:48:12.028405 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 17:48:12.028545 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":25,"spinlock_wait_cycles":896,"thread_start_us":77,"threads_started":1}
I20260812 17:48:12.029891 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.021612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45255 (local address 127.0.0.1:39192)
0812 17:48:12.023081 (+ 1469us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:12.023099 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:12.023198 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:12.023941 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:12.023944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:12.023964 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:12.024085 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.024091 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.025605 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.025609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.027482 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.027495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.028888 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.028892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.029016 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.029052 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 17:48:12.029148 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 17:48:12.029760 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1396,"mutex_wait_us":553,"thread_start_us":97,"threads_started":1}
I20260812 17:48:12.033380 16739 tablet_service.cc:1467] Processing CreateTablet for tablet d16b8da2d387434598698321ddd2a5dd (DEFAULT_TABLE table=test-table [id=9281f3c2ddbc442ebd81cb6e59d3ebd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:12.033383 16991 tablet_service.cc:1467] Processing CreateTablet for tablet d16b8da2d387434598698321ddd2a5dd (DEFAULT_TABLE table=test-table [id=9281f3c2ddbc442ebd81cb6e59d3ebd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:12.033375 16521 tablet_service.cc:1467] Processing CreateTablet for tablet d16b8da2d387434598698321ddd2a5dd (DEFAULT_TABLE table=test-table [id=9281f3c2ddbc442ebd81cb6e59d3ebd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:12.034575 16521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d16b8da2d387434598698321ddd2a5dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:12.034570 16739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d16b8da2d387434598698321ddd2a5dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:12.034574 16991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d16b8da2d387434598698321ddd2a5dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:12.041718 17128 tablet_bootstrap.cc:492] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb: Bootstrap starting.
I20260812 17:48:12.043568 17130 tablet_bootstrap.cc:492] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516: Bootstrap starting.
I20260812 17:48:12.043742 17131 tablet_bootstrap.cc:492] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276: Bootstrap starting.
I20260812 17:48:12.045231 17128 tablet_bootstrap.cc:654] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:12.045526 17130 tablet_bootstrap.cc:654] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:12.045987 17131 tablet_bootstrap.cc:654] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:12.046257 17128 log.cc:824] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:12.046290 17130 log.cc:824] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:12.046869 17131 log.cc:824] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:12.048709 17128 tablet_bootstrap.cc:492] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb: No bootstrap required, opened a new log
I20260812 17:48:12.048967 17128 ts_tablet_manager.cc:1397] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 17:48:12.049304 17130 tablet_bootstrap.cc:492] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516: No bootstrap required, opened a new log
I20260812 17:48:12.049482 17131 tablet_bootstrap.cc:492] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276: No bootstrap required, opened a new log
I20260812 17:48:12.049482 17130 ts_tablet_manager.cc:1397] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:48:12.049741 17131 ts_tablet_manager.cc:1397] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 17:48:12.053591 17130 raft_consensus.cc:348] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } }
I20260812 17:48:12.053887 17130 raft_consensus.cc:374] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:12.053941 17130 raft_consensus.cc:729] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a2a8f42e3a495bab1692890e3c7516, State: Initialized, Role: FOLLOWER
I20260812 17:48:12.053999 17128 raft_consensus.cc:348] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } }
I20260812 17:48:12.054278 17128 raft_consensus.cc:374] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:12.054354 17128 raft_consensus.cc:729] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e71aec20074411be80a4d2e616c8bb, State: Initialized, Role: FOLLOWER
I20260812 17:48:12.054440 17130 consensus_queue.cc:260] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } }
I20260812 17:48:12.054822 17128 consensus_queue.cc:260] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } }
I20260812 17:48:12.054806 17131 raft_consensus.cc:348] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } }
I20260812 17:48:12.055047 17131 raft_consensus.cc:374] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:12.055114 17131 raft_consensus.cc:729] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25665375389942e08e17cb942b777276, State: Initialized, Role: FOLLOWER
I20260812 17:48:12.055409 17095 heartbeater.cc:503] Master 127.13.38.62:43187 was elected leader, sending a full tablet report...
I20260812 17:48:12.055611 17130 ts_tablet_manager.cc:1428] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 17:48:12.055505 17131 consensus_queue.cc:260] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } }
I20260812 17:48:12.055900 17128 ts_tablet_manager.cc:1428] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 17:48:12.056386 16629 heartbeater.cc:503] Master 127.13.38.62:43187 was elected leader, sending a full tablet report...
I20260812 17:48:12.056537 17131 ts_tablet_manager.cc:1428] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260812 17:48:12.056788 16861 heartbeater.cc:503] Master 127.13.38.62:43187 was elected leader, sending a full tablet report...
W20260812 17:48:12.212138 17097 tablet.cc:2369] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:48:12.221344 16862 tablet.cc:2369] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:48:12.227743 16631 tablet.cc:2369] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:48:12.332504 17140 raft_consensus.cc:482] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:12.332796 17140 raft_consensus.cc:504] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } }
I20260812 17:48:12.334182 17140 leader_election.cc:288] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25665375389942e08e17cb942b777276 (127.13.38.2:41289), b5e71aec20074411be80a4d2e616c8bb (127.13.38.1:45255)
I20260812 17:48:12.338569 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.334556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:41289 (local address 127.13.38.3:39527)
0812 17:48:12.334728 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:12.334749 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:12.334886 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:12.335287 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:12.335291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:12.335314 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:12.335465 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.335473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.336363 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.336366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.337488 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.337500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.338136 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.338140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.338298 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.338321 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 17:48:12.338386 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 17:48:12.338451 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 17:48:12.338667 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.334676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:39527 (local address 127.13.38.2:41289)
0812 17:48:12.334845 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.334850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.334869 (+ 19us) server_negotiation.cc:407] Connection header received
0812 17:48:12.335040 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.335044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.335108 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.335198 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:12.335620 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.336243 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.337627 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.338011 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.338102 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.338160 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.338426 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.338516 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 17:48:12.339458 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.335050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:56647 (local address 127.13.38.1:45255)
0812 17:48:12.335201 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.335206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.335452 (+ 246us) server_negotiation.cc:407] Connection header received
0812 17:48:12.335610 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.335613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.335666 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.335747 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:12.336480 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.336971 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.338274 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.338865 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.338967 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.339016 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.339258 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.339338 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 17:48:12.339498 17144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.334912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45255 (local address 127.13.38.3:56647)
0812 17:48:12.335343 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:12.335358 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:12.335485 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:12.335767 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:12.335769 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:12.335786 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:12.336342 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.336353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.337094 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.337098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.338132 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.338142 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.339017 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.339023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.339109 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.339130 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 17:48:12.339237 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 17:48:12.339314 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":1,"thread_start_us":191,"threads_started":1}
I20260812 17:48:12.339712 16779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d16b8da2d387434598698321ddd2a5dd" candidate_uuid: "27a2a8f42e3a495bab1692890e3c7516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25665375389942e08e17cb942b777276" is_pre_election: true
I20260812 17:48:12.340062 16779 raft_consensus.cc:2454] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a2a8f42e3a495bab1692890e3c7516 in term 0.
I20260812 17:48:12.340335 16552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d16b8da2d387434598698321ddd2a5dd" candidate_uuid: "27a2a8f42e3a495bab1692890e3c7516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e71aec20074411be80a4d2e616c8bb" is_pre_election: true
I20260812 17:48:12.340672 16915 leader_election.cc:302] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25665375389942e08e17cb942b777276, 27a2a8f42e3a495bab1692890e3c7516; no voters:
I20260812 17:48:12.340694 16552 raft_consensus.cc:2454] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a2a8f42e3a495bab1692890e3c7516 in term 0.
I20260812 17:48:12.341007 17140 raft_consensus.cc:2781] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:12.341130 17140 raft_consensus.cc:482] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:12.341202 17140 raft_consensus.cc:3037] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:12.342420 17140 raft_consensus.cc:504] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } }
I20260812 17:48:12.342830 17140 leader_election.cc:288] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [CANDIDATE]: Term 1 election: Requested vote from peers 25665375389942e08e17cb942b777276 (127.13.38.2:41289), b5e71aec20074411be80a4d2e616c8bb (127.13.38.1:45255)
I20260812 17:48:12.343308 16779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d16b8da2d387434598698321ddd2a5dd" candidate_uuid: "27a2a8f42e3a495bab1692890e3c7516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25665375389942e08e17cb942b777276"
I20260812 17:48:12.343394 16552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d16b8da2d387434598698321ddd2a5dd" candidate_uuid: "27a2a8f42e3a495bab1692890e3c7516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e71aec20074411be80a4d2e616c8bb"
I20260812 17:48:12.343474 16779 raft_consensus.cc:3037] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:12.343616 16552 raft_consensus.cc:3037] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:12.344700 16779 raft_consensus.cc:2454] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a2a8f42e3a495bab1692890e3c7516 in term 1.
I20260812 17:48:12.344786 16552 raft_consensus.cc:2454] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a2a8f42e3a495bab1692890e3c7516 in term 1.
I20260812 17:48:12.345165 16915 leader_election.cc:302] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a2a8f42e3a495bab1692890e3c7516, b5e71aec20074411be80a4d2e616c8bb; no voters:
I20260812 17:48:12.345396 17140 raft_consensus.cc:2781] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:12.345669 17140 raft_consensus.cc:686] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 1 LEADER]: Becoming Leader. State: Replica: 27a2a8f42e3a495bab1692890e3c7516, State: Running, Role: LEADER
I20260812 17:48:12.346076 17140 consensus_queue.cc:237] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } }
I20260812 17:48:12.349725 16307 catalog_manager.cc:5495] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3). New cstate: current_term: 1 leader_uuid: "27a2a8f42e3a495bab1692890e3c7516" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } health_report { overall_health: UNKNOWN } } }
Aug 12 17:48:12 dist-test-slave-sb2z krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556891, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:12.415088 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.405664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43187 (local address 127.0.0.1:47964)
0812 17:48:12.406035 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:12.406051 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:12.406196 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:12.406468 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:12.406470 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:12.406947 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:12.407054 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.407061 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.407989 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.407994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.408549 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.408557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.409035 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.409039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.409120 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.409704 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:12.409717 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:12.411362 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:12.413445 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:12.413455 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:12.413462 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:12.413743 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:12.414048 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:12.414052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:12.414054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:12.414150 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:12.414566 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:12.414573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:12.414699 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 17:48:12.414797 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 17:48:12.414939 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":147,"threads_started":1}
I20260812 17:48:12.415174 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.405749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47964 (local address 127.13.38.62:43187)
0812 17:48:12.405909 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.405914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.406148 (+ 234us) server_negotiation.cc:407] Connection header received
0812 17:48:12.406285 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.406287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.406331 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.406409 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:12.407176 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.407865 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.408676 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.408892 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.409016 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.411656 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:12.411659 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:12.411661 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:12.411687 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:12.413299 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:12.413843 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:12.413845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:12.413846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:12.413927 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:12.414250 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:12.414252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:12.414253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:12.414441 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:12.414521 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.414860 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.415018 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":65}
I20260812 17:48:12.419512 16307 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:47964:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 17:48:12.420462 16307 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:12.425619 16521 tablet_service.cc:1467] Processing CreateTablet for tablet cc2754ea9c194148a02b4f53f88ffef6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f32005473a7649f282f32372fbc8eff2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 17:48:12.425618 16739 tablet_service.cc:1467] Processing CreateTablet for tablet cc2754ea9c194148a02b4f53f88ffef6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f32005473a7649f282f32372fbc8eff2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 17:48:12.425783 16991 tablet_service.cc:1467] Processing CreateTablet for tablet cc2754ea9c194148a02b4f53f88ffef6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f32005473a7649f282f32372fbc8eff2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 17:48:12.426487 16739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc2754ea9c194148a02b4f53f88ffef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:12.426487 16521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc2754ea9c194148a02b4f53f88ffef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:12.426528 16991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc2754ea9c194148a02b4f53f88ffef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:12.429538 17130 tablet_bootstrap.cc:492] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: Bootstrap starting.
I20260812 17:48:12.429742 17131 tablet_bootstrap.cc:492] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276: Bootstrap starting.
I20260812 17:48:12.429909 17128 tablet_bootstrap.cc:492] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb: Bootstrap starting.
I20260812 17:48:12.430531 17130 tablet_bootstrap.cc:654] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:12.430886 17131 tablet_bootstrap.cc:654] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:12.430909 17128 tablet_bootstrap.cc:654] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:12.432344 17131 tablet_bootstrap.cc:492] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276: No bootstrap required, opened a new log
I20260812 17:48:12.432379 17128 tablet_bootstrap.cc:492] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb: No bootstrap required, opened a new log
I20260812 17:48:12.432441 17130 tablet_bootstrap.cc:492] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: No bootstrap required, opened a new log
I20260812 17:48:12.432461 17128 ts_tablet_manager.cc:1397] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:12.432467 17131 ts_tablet_manager.cc:1397] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 17:48:12.432536 17130 ts_tablet_manager.cc:1397] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:12.432965 17130 raft_consensus.cc:348] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.432999 17128 raft_consensus.cc:348] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.432960 17131 raft_consensus.cc:348] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.433101 17130 raft_consensus.cc:374] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:12.433101 17128 raft_consensus.cc:374] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:12.433117 17131 raft_consensus.cc:374] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:12.433136 17128 raft_consensus.cc:729] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e71aec20074411be80a4d2e616c8bb, State: Initialized, Role: FOLLOWER
I20260812 17:48:12.433136 17130 raft_consensus.cc:729] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a2a8f42e3a495bab1692890e3c7516, State: Initialized, Role: FOLLOWER
I20260812 17:48:12.433161 17131 raft_consensus.cc:729] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25665375389942e08e17cb942b777276, State: Initialized, Role: FOLLOWER
I20260812 17:48:12.433260 17128 consensus_queue.cc:260] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.433260 17130 consensus_queue.cc:260] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.433292 17131 consensus_queue.cc:260] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.433543 17130 ts_tablet_manager.cc:1428] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:12.433533 17128 ts_tablet_manager.cc:1428] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 17:48:12.433678 17142 tablet_replica.cc:439] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } }
I20260812 17:48:12.433833 17142 tablet_replica.cc:442] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:12.433820 17141 tablet_replica.cc:439] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } }
I20260812 17:48:12.433957 17141 tablet_replica.cc:442] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:12.434628 17131 ts_tablet_manager.cc:1428] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 17:48:12.434900 17140 tablet_replica.cc:439] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } }
I20260812 17:48:12.435057 17140 tablet_replica.cc:442] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:12.449107 17140 raft_consensus.cc:482] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:12.449271 17140 raft_consensus.cc:504] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.449795 17140 leader_election.cc:288] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5e71aec20074411be80a4d2e616c8bb (127.13.38.1:45255), 25665375389942e08e17cb942b777276 (127.13.38.2:41289)
I20260812 17:48:12.450384 16552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc2754ea9c194148a02b4f53f88ffef6" candidate_uuid: "27a2a8f42e3a495bab1692890e3c7516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e71aec20074411be80a4d2e616c8bb" is_pre_election: true
I20260812 17:48:12.450393 16779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc2754ea9c194148a02b4f53f88ffef6" candidate_uuid: "27a2a8f42e3a495bab1692890e3c7516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25665375389942e08e17cb942b777276" is_pre_election: true
I20260812 17:48:12.450608 16779 raft_consensus.cc:2454] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a2a8f42e3a495bab1692890e3c7516 in term 0.
I20260812 17:48:12.450613 16552 raft_consensus.cc:2454] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a2a8f42e3a495bab1692890e3c7516 in term 0.
I20260812 17:48:12.450977 16915 leader_election.cc:302] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25665375389942e08e17cb942b777276, 27a2a8f42e3a495bab1692890e3c7516; no voters:
I20260812 17:48:12.451211 17140 raft_consensus.cc:2781] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:12.451292 17140 raft_consensus.cc:482] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:12.451340 17140 raft_consensus.cc:3037] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:12.452411 17140 raft_consensus.cc:504] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.452768 17140 leader_election.cc:288] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [CANDIDATE]: Term 1 election: Requested vote from peers b5e71aec20074411be80a4d2e616c8bb (127.13.38.1:45255), 25665375389942e08e17cb942b777276 (127.13.38.2:41289)
I20260812 17:48:12.453238 16552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc2754ea9c194148a02b4f53f88ffef6" candidate_uuid: "27a2a8f42e3a495bab1692890e3c7516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e71aec20074411be80a4d2e616c8bb"
I20260812 17:48:12.453253 16779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc2754ea9c194148a02b4f53f88ffef6" candidate_uuid: "27a2a8f42e3a495bab1692890e3c7516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25665375389942e08e17cb942b777276"
I20260812 17:48:12.453395 16779 raft_consensus.cc:3037] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:12.453397 16552 raft_consensus.cc:3037] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:12.454684 16779 raft_consensus.cc:2454] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a2a8f42e3a495bab1692890e3c7516 in term 1.
I20260812 17:48:12.454684 16552 raft_consensus.cc:2454] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a2a8f42e3a495bab1692890e3c7516 in term 1.
I20260812 17:48:12.454993 16915 leader_election.cc:302] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25665375389942e08e17cb942b777276, 27a2a8f42e3a495bab1692890e3c7516; no voters:
I20260812 17:48:12.455246 17140 raft_consensus.cc:2781] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:12.455351 17140 raft_consensus.cc:686] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 LEADER]: Becoming Leader. State: Replica: 27a2a8f42e3a495bab1692890e3c7516, State: Running, Role: LEADER
I20260812 17:48:12.455488 17140 consensus_queue.cc:237] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.456358 17145 tablet_replica.cc:439] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: TxnStatusTablet state changed. Reason: New leader 27a2a8f42e3a495bab1692890e3c7516. Latest consensus state: current_term: 1 leader_uuid: "27a2a8f42e3a495bab1692890e3c7516" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } }
I20260812 17:48:12.456502 17145 tablet_replica.cc:442] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: This TxnStatusTablet replica's current role is: LEADER
I20260812 17:48:12.457077 17158 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 17:48:12.457342 17158 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 17:48:12.457688 16305 catalog_manager.cc:5495] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3). New cstate: current_term: 1 leader_uuid: "27a2a8f42e3a495bab1692890e3c7516" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } health_report { overall_health: HEALTHY } } }
I20260812 17:48:12.475090 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.470394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:44381 (local address 127.0.0.1:52904)
0812 17:48:12.470561 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:12.470584 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:12.470675 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:12.471043 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:12.471045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:12.471052 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:12.471149 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.471153 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.472263 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.472266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.473005 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.473016 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.473658 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.473662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.473750 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.473837 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:12.474761 (+ 924us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:12.474769 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 17:48:12.474842 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 17:48:12.474944 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 17:48:12.475099 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.470530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52904 (local address 127.13.38.3:44381)
0812 17:48:12.470651 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.470657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.470674 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:48:12.470827 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.470830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.470899 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.471028 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:12.471261 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.472132 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.473133 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.473523 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.473629 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.474029 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:12.474572 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:12.474666 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.474884 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.474952 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 17:48:12.482653 16779 raft_consensus.cc:1261] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Refusing update from remote peer 27a2a8f42e3a495bab1692890e3c7516: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:12.482653 16552 raft_consensus.cc:1261] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Refusing update from remote peer 27a2a8f42e3a495bab1692890e3c7516: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:12.483445 17140 consensus_queue.cc:1035] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:12.483660 17145 consensus_queue.cc:1035] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:12.488579 17141 tablet_replica.cc:439] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb: TxnStatusTablet state changed. Reason: New leader 27a2a8f42e3a495bab1692890e3c7516. Latest consensus state: current_term: 1 leader_uuid: "27a2a8f42e3a495bab1692890e3c7516" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } }
I20260812 17:48:12.488744 17141 tablet_replica.cc:442] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:12.489181 17142 tablet_replica.cc:439] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276: TxnStatusTablet state changed. Reason: New leader 27a2a8f42e3a495bab1692890e3c7516. Latest consensus state: current_term: 1 leader_uuid: "27a2a8f42e3a495bab1692890e3c7516" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } }
I20260812 17:48:12.489293 17142 tablet_replica.cc:442] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:12.494174 17164 mvcc.cc:205] Tried to move back new op lower bound from 7317737031601147904 to 7317737031499509760. Current Snapshot: MvccSnapshot[applied={T|T < 7317737031601147904}]
I20260812 17:48:12.494472 17142 tablet_replica.cc:439] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27a2a8f42e3a495bab1692890e3c7516" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } }
I20260812 17:48:12.494611 17142 tablet_replica.cc:442] T cc2754ea9c194148a02b4f53f88ffef6 P 25665375389942e08e17cb942b777276: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:12.494745 17163 mvcc.cc:205] Tried to move back new op lower bound from 7317737031601147904 to 7317737031499509760. Current Snapshot: MvccSnapshot[applied={T|T < 7317737031601147904}]
I20260812 17:48:12.495517 17141 tablet_replica.cc:439] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27a2a8f42e3a495bab1692890e3c7516" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } }
I20260812 17:48:12.495692 17141 tablet_replica.cc:442] T cc2754ea9c194148a02b4f53f88ffef6 P b5e71aec20074411be80a4d2e616c8bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:12.496268 17145 tablet_replica.cc:439] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27a2a8f42e3a495bab1692890e3c7516" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } }
I20260812 17:48:12.496409 17145 tablet_replica.cc:442] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: This TxnStatusTablet replica's current role is: LEADER
I20260812 17:48:12.496522 17140 tablet_replica.cc:439] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27a2a8f42e3a495bab1692890e3c7516" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } } }
I20260812 17:48:12.496621 17140 tablet_replica.cc:442] T cc2754ea9c194148a02b4f53f88ffef6 P 27a2a8f42e3a495bab1692890e3c7516: This TxnStatusTablet replica's current role is: LEADER
I20260812 17:48:12.509234 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.505426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52916 (local address 127.13.38.3:44381)
0812 17:48:12.505592 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.505596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.505743 (+ 147us) server_negotiation.cc:407] Connection header received
0812 17:48:12.505881 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.505884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.505930 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.506001 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:12.506287 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.506844 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.508145 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.508378 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.508483 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.508748 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:12.508819 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:12.508876 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.509009 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.509057 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 17:48:12.522233 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.518132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:46025 (local address 127.13.38.3:44381)
0812 17:48:12.518251 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.518255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.518466 (+ 211us) server_negotiation.cc:407] Connection header received
0812 17:48:12.518587 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.518589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.518650 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.518729 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:12.519058 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.519728 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.521117 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.521328 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.521411 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.521793 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:12.521864 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:12.521920 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.522063 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.522109 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":48}
I20260812 17:48:12.522301 17184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.518013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:44381 (local address 127.13.38.3:46025)
0812 17:48:12.518352 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:12.518367 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:12.518483 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:12.518762 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:12.518765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:12.518774 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:12.518940 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.518947 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.519843 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.519847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.520959 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.520971 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.521451 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.521455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.521567 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.521625 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:12.521955 (+ 330us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:12.521962 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 17:48:12.522038 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 17:48:12.522119 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":19,"thread_start_us":126,"threads_started":1}
I20260812 17:48:12.530799 16552 raft_consensus.cc:1261] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Refusing update from remote peer 27a2a8f42e3a495bab1692890e3c7516: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:12.530794 16779 raft_consensus.cc:1261] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Refusing update from remote peer 27a2a8f42e3a495bab1692890e3c7516: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:12.531266 17169 consensus_queue.cc:1035] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:12.531430 17145 consensus_queue.cc:1035] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:12.586812 16309 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:12.587037 16309 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:12.590062 16309 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538: Sending DeleteTablet for 3 replicas of tablet d16b8da2d387434598698321ddd2a5dd
I20260812 17:48:12.590942 16989 tablet_service.cc:1514] Processing DeleteTablet for tablet d16b8da2d387434598698321ddd2a5dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:12 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:52898
I20260812 17:48:12.591094 16521 tablet_service.cc:1514] Processing DeleteTablet for tablet d16b8da2d387434598698321ddd2a5dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:12 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:39192
I20260812 17:48:12.591331 16739 tablet_service.cc:1514] Processing DeleteTablet for tablet d16b8da2d387434598698321ddd2a5dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:12 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:33344
I20260812 17:48:12.591691 17189 tablet_replica.cc:330] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516: stopping tablet replica
I20260812 17:48:12.591898 17190 tablet_replica.cc:330] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb: stopping tablet replica
I20260812 17:48:12.592089 17189 raft_consensus.cc:2229] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:12.592406 17190 raft_consensus.cc:2229] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:12.592487 17189 raft_consensus.cc:2258] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:12.592586 16309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 17:48:12.592939 17190 raft_consensus.cc:2258] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 17:48:12.593079 16309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:12.594008 17189 ts_tablet_manager.cc:1905] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:12.594542 17190 ts_tablet_manager.cc:1905] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:12.597398 17189 ts_tablet_manager.cc:1918] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:12.597515 17189 log.cc:1196] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/d16b8da2d387434598698321ddd2a5dd
I20260812 17:48:12.597951 17190 ts_tablet_manager.cc:1918] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:12.598534 16521 tablet_service.cc:1467] Processing CreateTablet for tablet 54a661926ba148b3893dd314b7d329c3 (DEFAULT_TABLE table=test-table [id=b5b9e1a1386545979092c7b8b0d31a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:12.598681 16989 tablet_service.cc:1467] Processing CreateTablet for tablet 54a661926ba148b3893dd314b7d329c3 (DEFAULT_TABLE table=test-table [id=b5b9e1a1386545979092c7b8b0d31a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:12.598822 16521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54a661926ba148b3893dd314b7d329c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:12.598969 16989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54a661926ba148b3893dd314b7d329c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:12.599699 17191 tablet_replica.cc:330] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276: stopping tablet replica
I20260812 17:48:12.600096 17191 raft_consensus.cc:2229] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:12.600363 17191 raft_consensus.cc:2258] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:12.601727 17130 tablet_bootstrap.cc:492] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516: Bootstrap starting.
I20260812 17:48:12.601758 16739 tablet_service.cc:1467] Processing CreateTablet for tablet 54a661926ba148b3893dd314b7d329c3 (DEFAULT_TABLE table=test-table [id=b5b9e1a1386545979092c7b8b0d31a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:12.602075 16739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54a661926ba148b3893dd314b7d329c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:12.602152 17190 log.cc:1196] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/wal/wals/d16b8da2d387434598698321ddd2a5dd
I20260812 17:48:12.602574 17190 ts_tablet_manager.cc:1939] T d16b8da2d387434598698321ddd2a5dd P b5e71aec20074411be80a4d2e616c8bb: Deleting consensus metadata
I20260812 17:48:12.602877 17130 tablet_bootstrap.cc:654] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:12.604614 17130 tablet_bootstrap.cc:492] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516: No bootstrap required, opened a new log
I20260812 17:48:12.604717 17130 ts_tablet_manager.cc:1397] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:12.604806 17131 tablet_bootstrap.cc:492] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276: Bootstrap starting.
I20260812 17:48:12.605235 17128 tablet_bootstrap.cc:492] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb: Bootstrap starting.
I20260812 17:48:12.605187 17130 raft_consensus.cc:348] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.605309 17130 raft_consensus.cc:374] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:12.605358 17130 raft_consensus.cc:729] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a2a8f42e3a495bab1692890e3c7516, State: Initialized, Role: FOLLOWER
I20260812 17:48:12.605501 17130 consensus_queue.cc:260] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.605574 17191 ts_tablet_manager.cc:1905] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:12.605769 17130 ts_tablet_manager.cc:1428] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:12.605878 17131 tablet_bootstrap.cc:654] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:12.606252 17128 tablet_bootstrap.cc:654] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:12.607198 17189 ts_tablet_manager.cc:1939] T d16b8da2d387434598698321ddd2a5dd P 27a2a8f42e3a495bab1692890e3c7516: Deleting consensus metadata
I20260812 17:48:12.607903 16289 catalog_manager.cc:4841] TS b5e71aec20074411be80a4d2e616c8bb (127.13.38.1:45255): tablet d16b8da2d387434598698321ddd2a5dd (table test-table [id=9281f3c2ddbc442ebd81cb6e59d3ebd6]) successfully deleted
I20260812 17:48:12.608573 17131 tablet_bootstrap.cc:492] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276: No bootstrap required, opened a new log
I20260812 17:48:12.608672 17131 ts_tablet_manager.cc:1397] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260812 17:48:12.608803 16283 catalog_manager.cc:4841] TS 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3:44381): tablet d16b8da2d387434598698321ddd2a5dd (table test-table [id=9281f3c2ddbc442ebd81cb6e59d3ebd6]) successfully deleted
I20260812 17:48:12.609135 17131 raft_consensus.cc:348] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.609279 17131 raft_consensus.cc:374] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:12.609334 17131 raft_consensus.cc:729] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25665375389942e08e17cb942b777276, State: Initialized, Role: FOLLOWER
I20260812 17:48:12.609488 17131 consensus_queue.cc:260] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.609741 17128 tablet_bootstrap.cc:492] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb: No bootstrap required, opened a new log
I20260812 17:48:12.609771 17131 ts_tablet_manager.cc:1428] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:12.609835 17128 ts_tablet_manager.cc:1397] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 17:48:12.610291 17128 raft_consensus.cc:348] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.610416 17128 raft_consensus.cc:374] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:12.610464 17128 raft_consensus.cc:729] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e71aec20074411be80a4d2e616c8bb, State: Initialized, Role: FOLLOWER
I20260812 17:48:12.610590 17128 consensus_queue.cc:260] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.610886 17128 ts_tablet_manager.cc:1428] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:12.611307 17191 ts_tablet_manager.cc:1918] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:12.611457 17191 log.cc:1196] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/d16b8da2d387434598698321ddd2a5dd
I20260812 17:48:12.611927 17191 ts_tablet_manager.cc:1939] T d16b8da2d387434598698321ddd2a5dd P 25665375389942e08e17cb942b777276: Deleting consensus metadata
I20260812 17:48:12.613145 16289 catalog_manager.cc:4841] TS 25665375389942e08e17cb942b777276 (127.13.38.2:41289): tablet d16b8da2d387434598698321ddd2a5dd (table test-table [id=9281f3c2ddbc442ebd81cb6e59d3ebd6]) successfully deleted
I20260812 17:48:12.697741 17141 raft_consensus.cc:482] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:12.697909 17141 raft_consensus.cc:504] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.699261 17141 leader_election.cc:288] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25665375389942e08e17cb942b777276 (127.13.38.2:41289), 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3:44381)
I20260812 17:48:12.704505 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.699626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:42735 (local address 127.13.38.2:41289)
0812 17:48:12.700024 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.700032 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.700052 (+ 20us) server_negotiation.cc:407] Connection header received
0812 17:48:12.700245 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.700251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.700316 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.700367 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:12.700838 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.701719 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.703332 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.703872 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.704018 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.704089 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.704289 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.704382 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286}
I20260812 17:48:12.704519 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.699506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:41289 (local address 127.13.38.1:42735)
0812 17:48:12.699957 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:12.699975 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:12.700101 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:12.700548 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:12.700551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:12.700569 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:12.700703 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.700710 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.701847 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.701851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.703210 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.703221 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.704046 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.704050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.704158 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.704178 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 17:48:12.704270 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 17:48:12.704345 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":133,"threads_started":1}
I20260812 17:48:12.704967 16779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a661926ba148b3893dd314b7d329c3" candidate_uuid: "b5e71aec20074411be80a4d2e616c8bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25665375389942e08e17cb942b777276" is_pre_election: true
I20260812 17:48:12.705143 16779 raft_consensus.cc:2454] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e71aec20074411be80a4d2e616c8bb in term 0.
I20260812 17:48:12.705374 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.699983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:40723 (local address 127.13.38.3:44381)
0812 17:48:12.700104 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.700108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.700329 (+ 221us) server_negotiation.cc:407] Connection header received
0812 17:48:12.700447 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.700451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.700504 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.700562 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:12.701007 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.701733 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.703707 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.704317 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.704420 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.704503 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.705174 (+ 671us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.705252 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 17:48:12.705574 16447 leader_election.cc:302] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25665375389942e08e17cb942b777276, b5e71aec20074411be80a4d2e616c8bb; no voters:
I20260812 17:48:12.705780 17197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.699578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:44381 (local address 127.13.38.1:40723)
0812 17:48:12.700230 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:12.700244 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:12.700343 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:12.700705 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:12.700707 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:12.700722 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:12.700845 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.700851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.702533 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.702536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.703518 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.703584 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.704900 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.704904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.705023 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.705047 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 17:48:12.705572 (+ 525us) client_negotiation.cc:240] Negotiation successful
0812 17:48:12.705643 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":97,"threads_started":1}
I20260812 17:48:12.705881 17141 raft_consensus.cc:2781] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:12.705966 17141 raft_consensus.cc:482] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:12.706013 17141 raft_consensus.cc:3037] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:12.706341 17024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a661926ba148b3893dd314b7d329c3" candidate_uuid: "b5e71aec20074411be80a4d2e616c8bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a2a8f42e3a495bab1692890e3c7516" is_pre_election: true
I20260812 17:48:12.706568 17024 raft_consensus.cc:2454] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e71aec20074411be80a4d2e616c8bb in term 0.
I20260812 17:48:12.707218 17141 raft_consensus.cc:504] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.707664 17141 leader_election.cc:288] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [CANDIDATE]: Term 1 election: Requested vote from peers 25665375389942e08e17cb942b777276 (127.13.38.2:41289), 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3:44381)
I20260812 17:48:12.707998 16779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a661926ba148b3893dd314b7d329c3" candidate_uuid: "b5e71aec20074411be80a4d2e616c8bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25665375389942e08e17cb942b777276"
I20260812 17:48:12.708038 17024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a661926ba148b3893dd314b7d329c3" candidate_uuid: "b5e71aec20074411be80a4d2e616c8bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a2a8f42e3a495bab1692890e3c7516"
I20260812 17:48:12.708127 16779 raft_consensus.cc:3037] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:12.708200 17024 raft_consensus.cc:3037] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:12.709161 17024 raft_consensus.cc:2454] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5e71aec20074411be80a4d2e616c8bb in term 1.
I20260812 17:48:12.709264 16779 raft_consensus.cc:2454] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5e71aec20074411be80a4d2e616c8bb in term 1.
I20260812 17:48:12.709488 16446 leader_election.cc:302] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a2a8f42e3a495bab1692890e3c7516, b5e71aec20074411be80a4d2e616c8bb; no voters:
I20260812 17:48:12.709754 17141 raft_consensus.cc:2781] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:12.710021 17141 raft_consensus.cc:686] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 1 LEADER]: Becoming Leader. State: Replica: b5e71aec20074411be80a4d2e616c8bb, State: Running, Role: LEADER
I20260812 17:48:12.710299 17141 consensus_queue.cc:237] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:12.714071 16309 catalog_manager.cc:5495] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb reported cstate change: term changed from 0 to 1, leader changed from <none> to b5e71aec20074411be80a4d2e616c8bb (127.13.38.1). New cstate: current_term: 1 leader_uuid: "b5e71aec20074411be80a4d2e616c8bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } health_report { overall_health: UNKNOWN } } }
I20260812 17:48:12.740967 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.736615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39208 (local address 127.13.38.1:45255)
0812 17:48:12.736733 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.736738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.736794 (+ 56us) server_negotiation.cc:407] Connection header received
0812 17:48:12.736996 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.736999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.737068 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.737193 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:12.737456 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.738235 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.739120 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.739473 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.739687 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.739955 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:12.740528 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:12.740654 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.740743 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.740806 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 17:48:12.753155 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.749177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:57791 (local address 127.13.38.3:44381)
0812 17:48:12.749379 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.749384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.749491 (+ 107us) server_negotiation.cc:407] Connection header received
0812 17:48:12.749629 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.749632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.749687 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.749770 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:12.750152 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.750881 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.751850 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.752156 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.752273 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.752574 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:12.752666 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:12.752750 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.752933 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.752995 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260812 17:48:12.753201 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.749014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:44381 (local address 127.13.38.1:57791)
0812 17:48:12.749386 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:12.749402 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:12.749528 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:12.749847 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:12.749850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:12.749858 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:12.750028 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.750034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.751014 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.751016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.751727 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.751738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.752292 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.752295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.752372 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.752425 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:12.752795 (+ 370us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:12.752804 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 17:48:12.752932 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 17:48:12.753011 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":128,"threads_started":1}
I20260812 17:48:12.760056 16779 raft_consensus.cc:1261] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Refusing update from remote peer b5e71aec20074411be80a4d2e616c8bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:12.760299 17024 raft_consensus.cc:1261] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Refusing update from remote peer b5e71aec20074411be80a4d2e616c8bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:12.760627 17141 consensus_queue.cc:1035] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:12.760824 17199 consensus_queue.cc:1035] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:12.765043 17162 mvcc.cc:205] Tried to move back new op lower bound from 7317737032740450304 to 7317737032544239616. Current Snapshot: MvccSnapshot[applied={T|T < 7317737032740450304}]
I20260812 17:48:12.781461 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.778097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:59225 (local address 127.13.38.1:45255)
0812 17:48:12.778216 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:12.778223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:12.778238 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:48:12.778283 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:12.778286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:12.778342 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:12.778424 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:12.778676 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.779196 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.780153 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.780466 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.780596 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.780927 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:12.781022 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:12.781095 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:12.781241 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 17:48:12.781305 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 17:48:12.781477 17184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:12.777770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45255 (local address 127.13.38.3:59225)
0812 17:48:12.778065 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:12.778082 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:12.778170 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:12.778438 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:12.778441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:12.778449 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:12.778553 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.778559 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.779320 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.779324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.780020 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:12.780030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:12.780600 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:12.780604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:12.780700 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:12.780740 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:12.781139 (+ 399us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:12.781146 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 17:48:12.781216 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 17:48:12.781325 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
W20260812 17:48:12.782529 17164 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 17:48:14.348299 16305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:14.348485 16305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:14.351219 16305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538: Sending DeleteTablet for 3 replicas of tablet 54a661926ba148b3893dd314b7d329c3
I20260812 17:48:14.351913 16521 tablet_service.cc:1514] Processing DeleteTablet for tablet 54a661926ba148b3893dd314b7d329c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:14 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:39192
I20260812 17:48:14.352049 16989 tablet_service.cc:1514] Processing DeleteTablet for tablet 54a661926ba148b3893dd314b7d329c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:14 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:52898
I20260812 17:48:14.352085 16739 tablet_service.cc:1514] Processing DeleteTablet for tablet 54a661926ba148b3893dd314b7d329c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:14 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:33344
I20260812 17:48:14.352384 17253 tablet_replica.cc:330] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb: stopping tablet replica
I20260812 17:48:14.352421 17256 tablet_replica.cc:330] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516: stopping tablet replica
I20260812 17:48:14.352444 17257 tablet_replica.cc:330] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276: stopping tablet replica
I20260812 17:48:14.352581 17256 raft_consensus.cc:2229] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:14.352615 17253 raft_consensus.cc:2229] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:14.352571 17257 raft_consensus.cc:2229] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:14.352725 17256 raft_consensus.cc:2258] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:14.352859 17257 raft_consensus.cc:2258] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:14.353212 17253 raft_consensus.cc:2258] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:14.353434 17256 ts_tablet_manager.cc:1905] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:14.353943 17253 ts_tablet_manager.cc:1905] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:14.356523 17256 ts_tablet_manager.cc:1918] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 17:48:14.356630 17256 log.cc:1196] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/54a661926ba148b3893dd314b7d329c3
I20260812 17:48:14.356925 17256 ts_tablet_manager.cc:1939] T 54a661926ba148b3893dd314b7d329c3 P 27a2a8f42e3a495bab1692890e3c7516: Deleting consensus metadata
I20260812 17:48:14.357237 17253 ts_tablet_manager.cc:1918] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 17:48:14.357252 17257 ts_tablet_manager.cc:1905] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:14.357422 17253 log.cc:1196] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/wal/wals/54a661926ba148b3893dd314b7d329c3
I20260812 17:48:14.357823 17253 ts_tablet_manager.cc:1939] T 54a661926ba148b3893dd314b7d329c3 P b5e71aec20074411be80a4d2e616c8bb: Deleting consensus metadata
I20260812 17:48:14.357824 16283 catalog_manager.cc:4841] TS 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3:44381): tablet 54a661926ba148b3893dd314b7d329c3 (table test-table [id=b5b9e1a1386545979092c7b8b0d31a91]) successfully deleted
I20260812 17:48:14.358669 16289 catalog_manager.cc:4841] TS b5e71aec20074411be80a4d2e616c8bb (127.13.38.1:45255): tablet 54a661926ba148b3893dd314b7d329c3 (table test-table [id=b5b9e1a1386545979092c7b8b0d31a91]) successfully deleted
I20260812 17:48:14.360726 17257 ts_tablet_manager.cc:1918] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 17:48:14.360827 17257 log.cc:1196] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/54a661926ba148b3893dd314b7d329c3
I20260812 17:48:14.361117 17257 ts_tablet_manager.cc:1939] T 54a661926ba148b3893dd314b7d329c3 P 25665375389942e08e17cb942b777276: Deleting consensus metadata
I20260812 17:48:14.361850 16289 catalog_manager.cc:4841] TS 25665375389942e08e17cb942b777276 (127.13.38.2:41289): tablet 54a661926ba148b3893dd314b7d329c3 (table test-table [id=b5b9e1a1386545979092c7b8b0d31a91]) successfully deleted
Aug 12 17:48:14 dist-test-slave-sb2z krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556894, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 17:48:14 dist-test-slave-sb2z krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556894, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:14.385718 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:14.376544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47974 (local address 127.13.38.62:43187)
0812 17:48:14.376868 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:14.376876 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:14.376892 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:48:14.376938 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:14.376940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:14.376985 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:14.377082 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:14.377899 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.378450 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.379399 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.379740 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.379842 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:14.382540 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:14.382543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:14.382545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:14.382576 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:14.384090 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:14.384523 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:14.384526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:14.384527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:14.384608 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:14.384852 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:14.384854 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:14.384856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:14.385053 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:14.385160 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:14.385412 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 17:48:14.385551 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":127,"threads_started":1}
I20260812 17:48:14.388841 16305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:48:14.389380 16305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:14.394995 16521 tablet_service.cc:1467] Processing CreateTablet for tablet f2816ef868724f0dad7fc85cf040f3d4 (DEFAULT_TABLE table=test-table [id=19a236784e454b3ba1edf3d22202940a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:14.395062 16739 tablet_service.cc:1467] Processing CreateTablet for tablet f2816ef868724f0dad7fc85cf040f3d4 (DEFAULT_TABLE table=test-table [id=19a236784e454b3ba1edf3d22202940a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:14.395376 16521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2816ef868724f0dad7fc85cf040f3d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:14.395401 16739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2816ef868724f0dad7fc85cf040f3d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:14.395342 16989 tablet_service.cc:1467] Processing CreateTablet for tablet f2816ef868724f0dad7fc85cf040f3d4 (DEFAULT_TABLE table=test-table [id=19a236784e454b3ba1edf3d22202940a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:14.395670 16989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2816ef868724f0dad7fc85cf040f3d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:14.398260 17270 tablet_bootstrap.cc:492] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276: Bootstrap starting.
I20260812 17:48:14.398476 17271 tablet_bootstrap.cc:492] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516: Bootstrap starting.
I20260812 17:48:14.397735 17269 tablet_bootstrap.cc:492] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb: Bootstrap starting.
I20260812 17:48:14.399236 17270 tablet_bootstrap.cc:654] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:14.399448 17271 tablet_bootstrap.cc:654] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:14.399853 17269 tablet_bootstrap.cc:654] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:14.400616 17270 tablet_bootstrap.cc:492] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276: No bootstrap required, opened a new log
I20260812 17:48:14.400722 17270 ts_tablet_manager.cc:1397] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 17:48:14.401151 17269 tablet_bootstrap.cc:492] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb: No bootstrap required, opened a new log
I20260812 17:48:14.401263 17269 ts_tablet_manager.cc:1397] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 17:48:14.401643 17270 raft_consensus.cc:348] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.401688 17269 raft_consensus.cc:348] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.401784 17270 raft_consensus.cc:374] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:14.401808 17269 raft_consensus.cc:374] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:14.401830 17270 raft_consensus.cc:729] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25665375389942e08e17cb942b777276, State: Initialized, Role: FOLLOWER
I20260812 17:48:14.401847 17269 raft_consensus.cc:729] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e71aec20074411be80a4d2e616c8bb, State: Initialized, Role: FOLLOWER
I20260812 17:48:14.401968 17270 consensus_queue.cc:260] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.402009 17269 consensus_queue.cc:260] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.402375 17270 ts_tablet_manager.cc:1428] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 17:48:14.402292 17269 ts_tablet_manager.cc:1428] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 17:48:14.403254 17271 tablet_bootstrap.cc:492] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516: No bootstrap required, opened a new log
I20260812 17:48:14.403374 17271 ts_tablet_manager.cc:1397] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 17:48:14.403898 17271 raft_consensus.cc:348] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.404036 17271 raft_consensus.cc:374] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:14.404085 17271 raft_consensus.cc:729] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a2a8f42e3a495bab1692890e3c7516, State: Initialized, Role: FOLLOWER
I20260812 17:48:14.404218 17271 consensus_queue.cc:260] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.404533 17271 ts_tablet_manager.cc:1428] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:14.488968 17244 raft_consensus.cc:482] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:14.489166 17244 raft_consensus.cc:504] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.489675 17244 leader_election.cc:288] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25665375389942e08e17cb942b777276 (127.13.38.2:41289), 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3:44381)
I20260812 17:48:14.490406 17024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2816ef868724f0dad7fc85cf040f3d4" candidate_uuid: "b5e71aec20074411be80a4d2e616c8bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a2a8f42e3a495bab1692890e3c7516" is_pre_election: true
I20260812 17:48:14.490381 16779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2816ef868724f0dad7fc85cf040f3d4" candidate_uuid: "b5e71aec20074411be80a4d2e616c8bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25665375389942e08e17cb942b777276" is_pre_election: true
I20260812 17:48:14.490621 16779 raft_consensus.cc:2454] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e71aec20074411be80a4d2e616c8bb in term 0.
I20260812 17:48:14.490633 17024 raft_consensus.cc:2454] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e71aec20074411be80a4d2e616c8bb in term 0.
I20260812 17:48:14.491142 16446 leader_election.cc:302] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a2a8f42e3a495bab1692890e3c7516, b5e71aec20074411be80a4d2e616c8bb; no voters:
I20260812 17:48:14.491380 17244 raft_consensus.cc:2781] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:14.491487 17244 raft_consensus.cc:482] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:14.491587 17244 raft_consensus.cc:3037] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:14.492738 17244 raft_consensus.cc:504] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.493144 17244 leader_election.cc:288] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [CANDIDATE]: Term 1 election: Requested vote from peers 25665375389942e08e17cb942b777276 (127.13.38.2:41289), 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3:44381)
I20260812 17:48:14.493552 16779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2816ef868724f0dad7fc85cf040f3d4" candidate_uuid: "b5e71aec20074411be80a4d2e616c8bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25665375389942e08e17cb942b777276"
I20260812 17:48:14.493587 17024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2816ef868724f0dad7fc85cf040f3d4" candidate_uuid: "b5e71aec20074411be80a4d2e616c8bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a2a8f42e3a495bab1692890e3c7516"
I20260812 17:48:14.493702 16779 raft_consensus.cc:3037] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:14.493726 17024 raft_consensus.cc:3037] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:14.494675 16779 raft_consensus.cc:2454] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5e71aec20074411be80a4d2e616c8bb in term 1.
I20260812 17:48:14.494791 17024 raft_consensus.cc:2454] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5e71aec20074411be80a4d2e616c8bb in term 1.
I20260812 17:48:14.495004 16447 leader_election.cc:302] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25665375389942e08e17cb942b777276, b5e71aec20074411be80a4d2e616c8bb; no voters:
I20260812 17:48:14.495203 17244 raft_consensus.cc:2781] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:14.495325 17244 raft_consensus.cc:686] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 1 LEADER]: Becoming Leader. State: Replica: b5e71aec20074411be80a4d2e616c8bb, State: Running, Role: LEADER
I20260812 17:48:14.495471 17244 consensus_queue.cc:237] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.497326 16307 catalog_manager.cc:5495] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb reported cstate change: term changed from 0 to 1, leader changed from <none> to b5e71aec20074411be80a4d2e616c8bb (127.13.38.1). New cstate: current_term: 1 leader_uuid: "b5e71aec20074411be80a4d2e616c8bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } health_report { overall_health: UNKNOWN } } }
I20260812 17:48:14.533385 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:14.530349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39218 (local address 127.13.38.1:45255)
0812 17:48:14.530624 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:14.530631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:14.530644 (+ 13us) server_negotiation.cc:407] Connection header received
0812 17:48:14.530686 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:14.530688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:14.530729 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:14.530805 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:14.531045 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.531589 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.532378 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.532600 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.532677 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:14.532947 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:14.533038 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:14.533106 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:14.533214 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 17:48:14.533259 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":120,"threads_started":1}
I20260812 17:48:14.540258 16779 raft_consensus.cc:1261] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Refusing update from remote peer b5e71aec20074411be80a4d2e616c8bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:14.540258 17024 raft_consensus.cc:1261] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Refusing update from remote peer b5e71aec20074411be80a4d2e616c8bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:14.540673 17222 consensus_queue.cc:1035] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:14.540854 17244 consensus_queue.cc:1035] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:14.574612 16307 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:14.574798 16307 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:14.576893 16307 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538: Sending DeleteTablet for 3 replicas of tablet f2816ef868724f0dad7fc85cf040f3d4
I20260812 17:48:14.577445 16521 tablet_service.cc:1514] Processing DeleteTablet for tablet f2816ef868724f0dad7fc85cf040f3d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:14 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:39192
I20260812 17:48:14.577569 16989 tablet_service.cc:1514] Processing DeleteTablet for tablet f2816ef868724f0dad7fc85cf040f3d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:14 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:52898
I20260812 17:48:14.577653 17253 tablet_replica.cc:330] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb: stopping tablet replica
I20260812 17:48:14.577657 16739 tablet_service.cc:1514] Processing DeleteTablet for tablet f2816ef868724f0dad7fc85cf040f3d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:14 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:33344
I20260812 17:48:14.577775 17256 tablet_replica.cc:330] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516: stopping tablet replica
I20260812 17:48:14.577791 17253 raft_consensus.cc:2229] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:14.577848 17257 tablet_replica.cc:330] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276: stopping tablet replica
I20260812 17:48:14.577914 17256 raft_consensus.cc:2229] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:14.577951 17257 raft_consensus.cc:2229] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:14.577997 17253 raft_consensus.cc:2258] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:14.578051 17256 raft_consensus.cc:2258] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:14.578071 17257 raft_consensus.cc:2258] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:14.578845 17253 ts_tablet_manager.cc:1905] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:14.579035 16307 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 17:48:14.579129 17257 ts_tablet_manager.cc:1905] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 17:48:14.579471 16307 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:14.581710 17257 ts_tablet_manager.cc:1918] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:14.581710 17253 ts_tablet_manager.cc:1918] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:14.581805 17257 log.cc:1196] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/f2816ef868724f0dad7fc85cf040f3d4
I20260812 17:48:14.581806 17253 log.cc:1196] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-0/wal/wals/f2816ef868724f0dad7fc85cf040f3d4
I20260812 17:48:14.582091 17257 ts_tablet_manager.cc:1939] T f2816ef868724f0dad7fc85cf040f3d4 P 25665375389942e08e17cb942b777276: Deleting consensus metadata
I20260812 17:48:14.582099 17253 ts_tablet_manager.cc:1939] T f2816ef868724f0dad7fc85cf040f3d4 P b5e71aec20074411be80a4d2e616c8bb: Deleting consensus metadata
I20260812 17:48:14.582857 17256 ts_tablet_manager.cc:1905] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:14.583709 16289 catalog_manager.cc:4841] TS b5e71aec20074411be80a4d2e616c8bb (127.13.38.1:45255): tablet f2816ef868724f0dad7fc85cf040f3d4 (table test-table [id=19a236784e454b3ba1edf3d22202940a]) successfully deleted
I20260812 17:48:14.584434 16289 catalog_manager.cc:4841] TS 25665375389942e08e17cb942b777276 (127.13.38.2:41289): tablet f2816ef868724f0dad7fc85cf040f3d4 (table test-table [id=19a236784e454b3ba1edf3d22202940a]) successfully deleted
I20260812 17:48:14.584470 16521 tablet_service.cc:1467] Processing CreateTablet for tablet 407b8ba7ab8242dbb63d5273751bc922 (DEFAULT_TABLE table=test-table [id=1fa331d599284f23b42db9d0e50ec379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:14.584749 16521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 407b8ba7ab8242dbb63d5273751bc922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:14.584715 16739 tablet_service.cc:1467] Processing CreateTablet for tablet 407b8ba7ab8242dbb63d5273751bc922 (DEFAULT_TABLE table=test-table [id=1fa331d599284f23b42db9d0e50ec379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:14.584940 16739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 407b8ba7ab8242dbb63d5273751bc922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:14.585616 17256 ts_tablet_manager.cc:1918] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:14.585677 17256 log.cc:1196] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/f2816ef868724f0dad7fc85cf040f3d4
I20260812 17:48:14.585872 17256 ts_tablet_manager.cc:1939] T f2816ef868724f0dad7fc85cf040f3d4 P 27a2a8f42e3a495bab1692890e3c7516: Deleting consensus metadata
I20260812 17:48:14.587203 17269 tablet_bootstrap.cc:492] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb: Bootstrap starting.
I20260812 17:48:14.587203 17270 tablet_bootstrap.cc:492] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276: Bootstrap starting.
I20260812 17:48:14.587386 16283 catalog_manager.cc:4841] TS 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3:44381): tablet f2816ef868724f0dad7fc85cf040f3d4 (table test-table [id=19a236784e454b3ba1edf3d22202940a]) successfully deleted
I20260812 17:48:14.587996 16989 tablet_service.cc:1467] Processing CreateTablet for tablet 407b8ba7ab8242dbb63d5273751bc922 (DEFAULT_TABLE table=test-table [id=1fa331d599284f23b42db9d0e50ec379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:14.588208 16989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 407b8ba7ab8242dbb63d5273751bc922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:14.588413 17269 tablet_bootstrap.cc:654] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:14.588423 17270 tablet_bootstrap.cc:654] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:14.589771 17270 tablet_bootstrap.cc:492] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276: No bootstrap required, opened a new log
I20260812 17:48:14.589888 17270 ts_tablet_manager.cc:1397] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:14.590165 17269 tablet_bootstrap.cc:492] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb: No bootstrap required, opened a new log
I20260812 17:48:14.590268 17269 ts_tablet_manager.cc:1397] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:14.590468 17270 raft_consensus.cc:348] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.590550 17271 tablet_bootstrap.cc:492] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516: Bootstrap starting.
I20260812 17:48:14.590607 17270 raft_consensus.cc:374] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:14.590647 17270 raft_consensus.cc:729] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25665375389942e08e17cb942b777276, State: Initialized, Role: FOLLOWER
I20260812 17:48:14.590714 17269 raft_consensus.cc:348] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.590827 17269 raft_consensus.cc:374] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:14.590869 17269 raft_consensus.cc:729] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e71aec20074411be80a4d2e616c8bb, State: Initialized, Role: FOLLOWER
I20260812 17:48:14.590835 17270 consensus_queue.cc:260] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.591022 17269 consensus_queue.cc:260] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.591321 17269 ts_tablet_manager.cc:1428] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 17:48:14.591598 17271 tablet_bootstrap.cc:654] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:14.591884 17270 ts_tablet_manager.cc:1428] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 17:48:14.592922 17271 tablet_bootstrap.cc:492] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516: No bootstrap required, opened a new log
I20260812 17:48:14.593017 17271 ts_tablet_manager.cc:1397] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 17:48:14.593477 17271 raft_consensus.cc:348] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.593617 17271 raft_consensus.cc:374] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:14.593655 17271 raft_consensus.cc:729] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a2a8f42e3a495bab1692890e3c7516, State: Initialized, Role: FOLLOWER
I20260812 17:48:14.593772 17271 consensus_queue.cc:260] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.594487 17271 ts_tablet_manager.cc:1428] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 17:48:14.684332 17283 raft_consensus.cc:482] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:14.684571 17283 raft_consensus.cc:504] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.685743 17283 leader_election.cc:288] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5e71aec20074411be80a4d2e616c8bb (127.13.38.1:45255), 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3:44381)
I20260812 17:48:14.690865 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:14.686088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:58699 (local address 127.13.38.1:45255)
0812 17:48:14.686245 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:14.686250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:14.686514 (+ 264us) server_negotiation.cc:407] Connection header received
0812 17:48:14.686683 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:14.686686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:14.686741 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:14.686832 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:14.687231 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.688080 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.689525 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.690181 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.690317 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:14.690394 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:14.690621 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 17:48:14.690702 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 17:48:14.690940 17297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:14.686022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45255 (local address 127.13.38.2:58699)
0812 17:48:14.686386 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:14.686402 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:14.686566 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:14.686859 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:14.686862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:14.686900 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:14.687120 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:14.687126 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.688229 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.688232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:14.689365 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:14.689377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.690333 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.690337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:14.690460 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:14.690483 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 17:48:14.690592 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 17:48:14.690679 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":176,"spinlock_wait_cycles":26368,"thread_start_us":98,"threads_started":1}
I20260812 17:48:14.691027 17298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:14.686062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:44381 (local address 127.13.38.2:56557)
0812 17:48:14.686462 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:14.686476 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:14.686571 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:14.686859 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:14.686862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:14.686901 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:14.687104 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:14.687113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.688214 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.688221 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:14.689397 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:14.689412 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.690492 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.690496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:14.690606 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:14.690638 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 17:48:14.690716 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 17:48:14.690801 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":139,"spinlock_wait_cycles":18816,"thread_start_us":110,"threads_started":1}
I20260812 17:48:14.691437 16552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "407b8ba7ab8242dbb63d5273751bc922" candidate_uuid: "25665375389942e08e17cb942b777276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e71aec20074411be80a4d2e616c8bb" is_pre_election: true
I20260812 17:48:14.691512 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:14.686131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:56557 (local address 127.13.38.3:44381)
0812 17:48:14.686425 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:14.686437 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:14.686563 (+ 126us) server_negotiation.cc:407] Connection header received
0812 17:48:14.686683 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:14.686686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:14.686739 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:14.686832 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:14.687231 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.688082 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.689579 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.690229 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.690360 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:14.690432 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:14.691260 (+ 828us) server_negotiation.cc:299] Negotiation successful
0812 17:48:14.691362 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":105,"threads_started":1}
I20260812 17:48:14.691661 16552 raft_consensus.cc:2454] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25665375389942e08e17cb942b777276 in term 0.
I20260812 17:48:14.692034 17024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "407b8ba7ab8242dbb63d5273751bc922" candidate_uuid: "25665375389942e08e17cb942b777276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a2a8f42e3a495bab1692890e3c7516" is_pre_election: true
I20260812 17:48:14.692087 16667 leader_election.cc:302] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25665375389942e08e17cb942b777276, b5e71aec20074411be80a4d2e616c8bb; no voters:
I20260812 17:48:14.692191 17024 raft_consensus.cc:2454] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25665375389942e08e17cb942b777276 in term 0.
I20260812 17:48:14.692483 17283 raft_consensus.cc:2781] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:14.692562 17283 raft_consensus.cc:482] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:14.692608 17283 raft_consensus.cc:3037] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:14.693753 17283 raft_consensus.cc:504] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.694142 17283 leader_election.cc:288] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [CANDIDATE]: Term 1 election: Requested vote from peers b5e71aec20074411be80a4d2e616c8bb (127.13.38.1:45255), 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3:44381)
I20260812 17:48:14.694523 17024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "407b8ba7ab8242dbb63d5273751bc922" candidate_uuid: "25665375389942e08e17cb942b777276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a2a8f42e3a495bab1692890e3c7516"
I20260812 17:48:14.694523 16552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "407b8ba7ab8242dbb63d5273751bc922" candidate_uuid: "25665375389942e08e17cb942b777276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e71aec20074411be80a4d2e616c8bb"
I20260812 17:48:14.694664 17024 raft_consensus.cc:3037] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:14.694664 16552 raft_consensus.cc:3037] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:14.695981 16552 raft_consensus.cc:2454] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25665375389942e08e17cb942b777276 in term 1.
I20260812 17:48:14.695971 17024 raft_consensus.cc:2454] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25665375389942e08e17cb942b777276 in term 1.
I20260812 17:48:14.696450 16667 leader_election.cc:302] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25665375389942e08e17cb942b777276, b5e71aec20074411be80a4d2e616c8bb; no voters:
I20260812 17:48:14.696650 17283 raft_consensus.cc:2781] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:14.696892 17283 raft_consensus.cc:686] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 1 LEADER]: Becoming Leader. State: Replica: 25665375389942e08e17cb942b777276, State: Running, Role: LEADER
I20260812 17:48:14.697189 17283 consensus_queue.cc:237] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } }
I20260812 17:48:14.699965 16307 catalog_manager.cc:5495] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25665375389942e08e17cb942b777276 (127.13.38.2). New cstate: current_term: 1 leader_uuid: "25665375389942e08e17cb942b777276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25665375389942e08e17cb942b777276" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 } health_report { overall_health: UNKNOWN } } }
I20260812 17:48:14.733167 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:14.728314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33350 (local address 127.13.38.2:41289)
0812 17:48:14.728630 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:14.728638 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:14.728654 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:48:14.728767 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:14.728770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:14.728836 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:14.729010 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:14.729244 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.730110 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.730937 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.731366 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.731500 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:14.731910 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:14.732617 (+ 707us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:14.732761 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:14.732849 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 17:48:14.732936 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":127,"threads_started":1}
I20260812 17:48:14.750583 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:14.746200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:34627 (local address 127.13.38.3:44381)
0812 17:48:14.746324 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:14.746333 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:14.746470 (+ 137us) server_negotiation.cc:407] Connection header received
0812 17:48:14.746583 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:14.746586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:14.746651 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:14.746737 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:14.747027 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.747901 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.749049 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.749456 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.749595 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:14.749924 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:14.750038 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:14.750132 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:14.750292 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 17:48:14.750379 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 17:48:14.750662 17307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:14.746012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:44381 (local address 127.13.38.2:34627)
0812 17:48:14.746362 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:14.746382 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:14.746481 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:14.746750 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:14.746753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:14.746762 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:14.746899 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:14.746905 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.748049 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.748052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:14.748905 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:14.748917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.749604 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.749607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:14.749711 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:14.749765 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:14.750151 (+ 386us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:14.750164 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 17:48:14.750270 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 17:48:14.750386 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":127,"threads_started":1}
I20260812 17:48:14.763764 16552 raft_consensus.cc:1261] T 407b8ba7ab8242dbb63d5273751bc922 P b5e71aec20074411be80a4d2e616c8bb [term 1 FOLLOWER]: Refusing update from remote peer 25665375389942e08e17cb942b777276: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:14.763767 17024 raft_consensus.cc:1261] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Refusing update from remote peer 25665375389942e08e17cb942b777276: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:14.764324 17300 consensus_queue.cc:1035] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5e71aec20074411be80a4d2e616c8bb" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:14.764544 17283 consensus_queue.cc:1035] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27a2a8f42e3a495bab1692890e3c7516" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:14.783130 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:14.779314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:41289 (local address 127.13.38.3:60337)
0812 17:48:14.779728 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:14.779747 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:14.779858 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:14.780153 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:14.780156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:14.780164 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:14.780269 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:14.780274 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.781061 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.781065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:14.781682 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:14.781690 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.782251 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.782254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:14.782344 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:14.782381 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:14.782814 (+ 433us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:14.782825 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 17:48:14.782899 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 17:48:14.782999 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":109,"threads_started":1}
I20260812 17:48:14.783226 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:14.779418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:60337 (local address 127.13.38.2:41289)
0812 17:48:14.779579 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:14.779585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:14.779827 (+ 242us) server_negotiation.cc:407] Connection header received
0812 17:48:14.779980 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:14.779983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:14.780039 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:14.780127 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:14.780380 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.780946 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.781808 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:14.782120 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:14.782251 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:14.782647 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:14.782722 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:14.782810 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:14.782988 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 17:48:14.783059 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 17:48:14.784276 17232 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 17:48:16.357635 16305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:16.357810 16305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:16.360728 16305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7938f86359c84b36938c86921e07e538: Sending DeleteTablet for 3 replicas of tablet 407b8ba7ab8242dbb63d5273751bc922
I20260812 17:48:16.361505 16521 tablet_service.cc:1514] Processing DeleteTablet for tablet 407b8ba7ab8242dbb63d5273751bc922 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:16 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:39192
I20260812 17:48:16.361552 16739 tablet_service.cc:1514] Processing DeleteTablet for tablet 407b8ba7ab8242dbb63d5273751bc922 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:16 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:33344
I20260812 17:48:16.361575 16989 tablet_service.cc:1514] Processing DeleteTablet for tablet 407b8ba7ab8242dbb63d5273751bc922 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:16 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:52898
I20260812 17:48:16.361931 17354 tablet_replica.cc:330] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516: stopping tablet replica
I20260812 17:48:16.362043 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 16411
I20260812 17:48:16.361932 17355 tablet_replica.cc:330] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276: stopping tablet replica
I20260812 17:48:16.362214 17355 raft_consensus.cc:2229] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:16.370899 17355 raft_consensus.cc:2258] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:16.371264 17354 raft_consensus.cc:2229] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:16.371515 17354 raft_consensus.cc:2258] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:16.371937 17355 ts_tablet_manager.cc:1905] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:16.373184 17354 ts_tablet_manager.cc:1905] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:16.375048 17355 ts_tablet_manager.cc:1918] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 17:48:16.375151 17355 log.cc:1196] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/407b8ba7ab8242dbb63d5273751bc922
I20260812 17:48:16.375496 17355 ts_tablet_manager.cc:1939] T 407b8ba7ab8242dbb63d5273751bc922 P 25665375389942e08e17cb942b777276: Deleting consensus metadata
I20260812 17:48:16.376309 17354 ts_tablet_manager.cc:1918] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 17:48:16.376470 17354 log.cc:1196] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/407b8ba7ab8242dbb63d5273751bc922
I20260812 17:48:16.376471 16289 catalog_manager.cc:4841] TS 25665375389942e08e17cb942b777276 (127.13.38.2:41289): tablet 407b8ba7ab8242dbb63d5273751bc922 (table test-table [id=1fa331d599284f23b42db9d0e50ec379]) successfully deleted
I20260812 17:48:16.376799 17354 ts_tablet_manager.cc:1939] T 407b8ba7ab8242dbb63d5273751bc922 P 27a2a8f42e3a495bab1692890e3c7516: Deleting consensus metadata
I20260812 17:48:16.377593 16283 catalog_manager.cc:4841] TS 27a2a8f42e3a495bab1692890e3c7516 (127.13.38.3:44381): tablet 407b8ba7ab8242dbb63d5273751bc922 (table test-table [id=1fa331d599284f23b42db9d0e50ec379]) successfully deleted
W20260812 17:48:16.382453 16289 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:45255 (error 108)
I20260812 17:48:16.382632 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 16635
I20260812 17:48:16.389382 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:16.388724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45255 (local address 127.0.0.1:39222)
0812 17:48:16.389142 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:16.389246 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:45255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":105,"threads_started":1}
W20260812 17:48:16.389665 16289 catalog_manager.cc:4584] TS b5e71aec20074411be80a4d2e616c8bb (127.13.38.1:45255): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 407b8ba7ab8242dbb63d5273751bc922: Network error: Client connection negotiation failed: client connection to 127.13.38.1:45255: connect: Connection refused (error 111)
I20260812 17:48:16.399629 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 16875
I20260812 17:48:16.413765 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 16255
2026-08-12T17:48:16Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6898 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:16 dist-test-slave-sb2z krb5kdc[17361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:16 dist-test-slave-sb2z krb5kdc[17361](info): set up 2 sockets
Aug 12 17:48:16 dist-test-slave-sb2z krb5kdc[17361](info): commencing operation
krb5kdc: starting...
W20260812 17:48:17.746191 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:17 dist-test-slave-sb2z krb5kdc[17361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:48:17.888127 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34561
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:34561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:18.007671 17384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:18.008059 17384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:18.008160 17384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:18.012385 17384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:18.012506 17384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:18.012544 17384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:18.012585 17384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:18.012622 17384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:18.018267 17384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:34561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:18.019572 17384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:18.020713 17384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:18.030624 17392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:18.030603 17391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:18.030663 17384 server_base.cc:1034] running on GCE node
W20260812 17:48:18.030597 17396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:18.031627 17384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:18.032828 17384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:18.034001 17384 hybrid_clock.cc:648] HybridClock initialized: now 1786556898033982 us; error 37 us; skew 500 ppm
Aug 12 17:48:18 dist-test-slave-sb2z krb5kdc[17361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556898, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:18.037210 17384 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:48:18.038256 17384 webserver.cc:460] Webserver started at http://127.13.38.62:37237/ using document root <none> and password file <none>
I20260812 17:48:18.038919 17384 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:18.039023 17384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:18.039285 17384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:18.041610 17384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "11c74678d5024ef28fb79a9c3fcaf77e"
format_stamp: "Formatted at 2026-08-12 17:48:18 on dist-test-slave-sb2z"
server_key: "e764b383995a51d0f459939ec3fc9bf6"
server_key_iv: "88b947c33ca7780b8d89b36351d4da3c"
server_key_version: "encryptionkey@0"
I20260812 17:48:18.042243 17384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "11c74678d5024ef28fb79a9c3fcaf77e"
format_stamp: "Formatted at 2026-08-12 17:48:18 on dist-test-slave-sb2z"
server_key: "e764b383995a51d0f459939ec3fc9bf6"
server_key_iv: "88b947c33ca7780b8d89b36351d4da3c"
server_key_version: "encryptionkey@0"
I20260812 17:48:18.046404 17384 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:48:18.049093 17406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:18.050495 17384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 17:48:18.050662 17384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "11c74678d5024ef28fb79a9c3fcaf77e"
format_stamp: "Formatted at 2026-08-12 17:48:18 on dist-test-slave-sb2z"
server_key: "e764b383995a51d0f459939ec3fc9bf6"
server_key_iv: "88b947c33ca7780b8d89b36351d4da3c"
server_key_version: "encryptionkey@0"
I20260812 17:48:18.050791 17384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:18.060873 17384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:18.064968 17384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:18.065208 17384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:18.073740 17384 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:34561
I20260812 17:48:18.073769 17503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:34561 every 8 connection(s)
I20260812 17:48:18.074937 17384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:18.076404 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 17384
I20260812 17:48:18.076536 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:18.076840 13464 external_mini_cluster.cc:1428] Setting key cd4e99a9b3707bfade73b9b4e9d6b1dc
I20260812 17:48:18.078486 17504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:18.085477 17504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e: Bootstrap starting.
Aug 12 17:48:18 dist-test-slave-sb2z krb5kdc[17361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:18.088327 17504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:18.089329 17504 log.cc:824] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:18.091629 17504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e: No bootstrap required, opened a new log
I20260812 17:48:18.094051 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:18.078303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38800 (local address 127.13.38.62:34561)
0812 17:48:18.078777 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:18.078801 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:18.078838 (+ 37us) server_negotiation.cc:407] Connection header received
0812 17:48:18.079503 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:18.079520 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:18.079933 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:18.080230 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:18.081206 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.082223 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.083452 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.084028 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.084154 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:18.087175 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:18.087181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:18.087198 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:18.087234 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:18.089403 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:18.089910 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:18.089919 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:18.089929 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:18.090111 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:18.091872 (+ 1761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:18.091876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:18.091877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:18.092345 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:18.092512 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:18.092870 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 17:48:18.093248 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":183,"threads_started":1}
I20260812 17:48:18.095996 17504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c74678d5024ef28fb79a9c3fcaf77e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34561 } }
I20260812 17:48:18.096261 17504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:18.096329 17504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c74678d5024ef28fb79a9c3fcaf77e, State: Initialized, Role: FOLLOWER
I20260812 17:48:18.096833 17504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c74678d5024ef28fb79a9c3fcaf77e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34561 } }
I20260812 17:48:18.096978 17504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:18.097050 17504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:18.097157 17504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:18.098136 17504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c74678d5024ef28fb79a9c3fcaf77e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34561 } }
I20260812 17:48:18.098495 17504 leader_election.cc:302] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11c74678d5024ef28fb79a9c3fcaf77e; no voters:
I20260812 17:48:18.098878 17504 leader_election.cc:288] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:18.099025 17516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:18.099282 17516 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [term 1 LEADER]: Becoming Leader. State: Replica: 11c74678d5024ef28fb79a9c3fcaf77e, State: Running, Role: LEADER
I20260812 17:48:18.099692 17516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c74678d5024ef28fb79a9c3fcaf77e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34561 } }
I20260812 17:48:18.100260 17504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:18.101619 17518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11c74678d5024ef28fb79a9c3fcaf77e. Latest consensus state: current_term: 1 leader_uuid: "11c74678d5024ef28fb79a9c3fcaf77e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c74678d5024ef28fb79a9c3fcaf77e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34561 } } }
I20260812 17:48:18.101660 17517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11c74678d5024ef28fb79a9c3fcaf77e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c74678d5024ef28fb79a9c3fcaf77e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34561 } } }
I20260812 17:48:18.101758 17518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:18.101763 17517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:18.102164 17532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:18.104969 17532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:18.110697 17532 catalog_manager.cc:1347] Generated new cluster ID: 8cfc356c74c048368ca4ad08f2756215
I20260812 17:48:18.110781 17532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:18.118424 17532 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:18.119762 17532 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:18.125799 17532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11c74678d5024ef28fb79a9c3fcaf77e: Generated new TSK 0
I20260812 17:48:18.126662 17532 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:48:18.164592 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34561
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:18.289528 17546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:18.289816 17546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:18.289862 17546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:18.294006 17546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:18.294071 17546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:18.294162 17546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:18.299450 17546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:18.300678 17546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:18.301872 17546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:18.313623 17554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:18.313644 17553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:18.313820 17546 server_base.cc:1034] running on GCE node
W20260812 17:48:18.313611 17556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:18.314447 17546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:18.315119 17546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:18.316293 17546 hybrid_clock.cc:648] HybridClock initialized: now 1786556898316252 us; error 49 us; skew 500 ppm
Aug 12 17:48:18 dist-test-slave-sb2z krb5kdc[17361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556898, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:18.319983 17546 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:48:18.320981 17546 webserver.cc:460] Webserver started at http://127.13.38.1:37933/ using document root <none> and password file <none>
I20260812 17:48:18.321612 17546 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:18.321712 17546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:18.321969 17546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:18.324236 17546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "7b9debd0844d4344a4f4c55b3150ccd0"
format_stamp: "Formatted at 2026-08-12 17:48:18 on dist-test-slave-sb2z"
server_key: "3821bb7e057fdd277b07b5281a57771d"
server_key_iv: "37207a15b1c05ea299ca47e77e2183ba"
server_key_version: "encryptionkey@0"
I20260812 17:48:18.324875 17546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "7b9debd0844d4344a4f4c55b3150ccd0"
format_stamp: "Formatted at 2026-08-12 17:48:18 on dist-test-slave-sb2z"
server_key: "3821bb7e057fdd277b07b5281a57771d"
server_key_iv: "37207a15b1c05ea299ca47e77e2183ba"
server_key_version: "encryptionkey@0"
I20260812 17:48:18.329015 17546 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:48:18.331800 17566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:18.333117 17546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:48:18.333357 17546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "7b9debd0844d4344a4f4c55b3150ccd0"
format_stamp: "Formatted at 2026-08-12 17:48:18 on dist-test-slave-sb2z"
server_key: "3821bb7e057fdd277b07b5281a57771d"
server_key_iv: "37207a15b1c05ea299ca47e77e2183ba"
server_key_version: "encryptionkey@0"
I20260812 17:48:18.333484 17546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:18.351902 17546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:18.355453 17546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:18.355726 17546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:18.356384 17546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:18.357282 17546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:18.357363 17546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:18.357437 17546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:18.357491 17546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:18.366474 17546 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:41057
I20260812 17:48:18.366497 17745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:41057 every 8 connection(s)
I20260812 17:48:18.367409 17546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:18.373433 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 17546
I20260812 17:48:18.373543 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:18.373858 13464 external_mini_cluster.cc:1428] Setting key 120b91542f55f70d512d9f02307d5d37
Aug 12 17:48:18 dist-test-slave-sb2z krb5kdc[17361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556898, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:18.382941 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:18.369358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:48963 (local address 127.13.38.62:34561)
0812 17:48:18.369516 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:18.369522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:18.370423 (+ 901us) server_negotiation.cc:407] Connection header received
0812 17:48:18.371679 (+ 1256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:18.371681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:18.371732 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:18.371801 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:18.373124 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.373749 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.375055 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.375404 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.375494 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:18.378786 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:18.378792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:18.378796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:18.378836 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:18.380478 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:18.381189 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:18.381192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:18.381194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:18.381270 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:18.381690 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:18.381692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:18.381693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:18.381881 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:18.381986 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:18.382701 (+ 715us) server_negotiation.cc:299] Negotiation successful
0812 17:48:18.382814 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 17:48:18.384299 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:18.369819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34561 (local address 127.13.38.1:48963)
0812 17:48:18.370265 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:18.370309 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:18.371456 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:18.372039 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:18.372046 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:18.372431 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:18.372951 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:18.372967 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.373894 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.373901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:18.374922 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:18.374933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.375595 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.375601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:18.375827 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:18.376505 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:18.376530 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:18.378595 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:18.380616 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:18.380626 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:18.380648 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:18.381082 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:18.381369 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:18.381372 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:18.381374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:18.381583 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:18.382002 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:18.382011 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:18.382396 (+ 385us) client_negotiation.cc:769] Sending connection context
0812 17:48:18.382666 (+ 270us) client_negotiation.cc:240] Negotiation successful
0812 17:48:18.383085 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:48:18.385754 17746 heartbeater.cc:347] Connected to a master server at 127.13.38.62:34561
I20260812 17:48:18.386063 17746 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:18.386624 17746 heartbeater.cc:511] Master 127.13.38.62:34561 requested a full tablet report, sending...
I20260812 17:48:18.388387 17446 ts_manager.cc:194] Registered new tserver with Master: 7b9debd0844d4344a4f4c55b3150ccd0 (127.13.38.1:41057)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
I20260812 17:48:18.389930 17446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:48963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:48:18.406726 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34561
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:18.524115 17754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:18.524436 17754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:18.524528 17754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:18.528633 17754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:18.528726 17754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:18.528815 17754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:18.534273 17754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:18.535490 17754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:18.536810 17754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:18.546905 17766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:18.546851 17769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:18.546974 17765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:18.547199 17754 server_base.cc:1034] running on GCE node
I20260812 17:48:18.547711 17754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:18.548463 17754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:18.549698 17754 hybrid_clock.cc:648] HybridClock initialized: now 1786556898549680 us; error 29 us; skew 500 ppm
Aug 12 17:48:18 dist-test-slave-sb2z krb5kdc[17361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556898, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:18.552990 17754 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:48:18.553944 17754 webserver.cc:460] Webserver started at http://127.13.38.2:34761/ using document root <none> and password file <none>
I20260812 17:48:18.554608 17754 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:18.554708 17754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:18.554975 17754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:18.557314 17754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "b43dd44410b44e6ca15e7a58ce6ca3bd"
format_stamp: "Formatted at 2026-08-12 17:48:18 on dist-test-slave-sb2z"
server_key: "85740902300fb9205ae1b0dcf3dc9efb"
server_key_iv: "6c32babce54690d50d43adf17ae722f8"
server_key_version: "encryptionkey@0"
I20260812 17:48:18.557952 17754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "b43dd44410b44e6ca15e7a58ce6ca3bd"
format_stamp: "Formatted at 2026-08-12 17:48:18 on dist-test-slave-sb2z"
server_key: "85740902300fb9205ae1b0dcf3dc9efb"
server_key_iv: "6c32babce54690d50d43adf17ae722f8"
server_key_version: "encryptionkey@0"
I20260812 17:48:18.562089 17754 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260812 17:48:18.564925 17783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:18.566372 17754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 17:48:18.566563 17754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "b43dd44410b44e6ca15e7a58ce6ca3bd"
format_stamp: "Formatted at 2026-08-12 17:48:18 on dist-test-slave-sb2z"
server_key: "85740902300fb9205ae1b0dcf3dc9efb"
server_key_iv: "6c32babce54690d50d43adf17ae722f8"
server_key_version: "encryptionkey@0"
I20260812 17:48:18.566700 17754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:18.586351 17754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:18.589988 17754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:18.590219 17754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:18.590920 17754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:18.591881 17754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:18.591954 17754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:18.592026 17754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:18.592082 17754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:18.602471 17754 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:41725
I20260812 17:48:18.602536 17978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:41725 every 8 connection(s)
I20260812 17:48:18.603473 17754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:18.604825 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 17754
I20260812 17:48:18.604939 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:18.605201 13464 external_mini_cluster.cc:1428] Setting key af5e23281a25930a70cb9af6d9f6b4d1
Aug 12 17:48:18 dist-test-slave-sb2z krb5kdc[17361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556898, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
I20260812 17:48:18.618928 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:18.605638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:37583 (local address 127.13.38.62:34561)
0812 17:48:18.605802 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:18.605807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:18.607086 (+ 1279us) server_negotiation.cc:407] Connection header received
0812 17:48:18.608168 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:18.608175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:18.608239 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:18.608338 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:18.609670 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.610254 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.611425 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.611716 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.611834 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:18.614739 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:18.614743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:18.614745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:18.614770 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:18.616382 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:18.616995 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:18.616999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:18.617003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:18.617152 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:18.617556 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:18.617559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:18.617563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:18.617803 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:18.617919 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:18.618555 (+ 636us) server_negotiation.cc:299] Negotiation successful
0812 17:48:18.618743 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 17:48:18.619941 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:18.606119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34561 (local address 127.13.38.2:37583)
0812 17:48:18.606877 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:18.606933 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:18.607958 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:18.608510 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:18.608516 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:18.608889 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:18.609517 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:18.609534 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.610389 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.610392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:18.611294 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:18.611304 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.611881 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.611885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:18.612069 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:18.612753 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:18.612771 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:18.614549 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:18.616523 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:18.616529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:18.616547 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:18.616878 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:18.617257 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:18.617260 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:18.617262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:18.617449 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:18.617922 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:18.617931 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:18.618289 (+ 358us) client_negotiation.cc:769] Sending connection context
0812 17:48:18.618508 (+ 219us) client_negotiation.cc:240] Negotiation successful
0812 17:48:18.618870 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":153,"threads_started":1}
I20260812 17:48:18.621266 17979 heartbeater.cc:347] Connected to a master server at 127.13.38.62:34561
I20260812 17:48:18.621534 17979 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:18.622076 17979 heartbeater.cc:511] Master 127.13.38.62:34561 requested a full tablet report, sending...
I20260812 17:48:18.623220 17446 ts_manager.cc:194] Registered new tserver with Master: b43dd44410b44e6ca15e7a58ce6ca3bd (127.13.38.2:41725)
I20260812 17:48:18.624177 17446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:37583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:48:18.638742 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34561
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:18.759430 17984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:18.759790 17984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:18.759837 17984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:18.764003 17984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:18.764077 17984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:18.764138 17984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:18.769549 17984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:18.770727 17984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:18.771896 17984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:18.782342 17995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:18.782353 17997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:18.782420 17992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:18.782608 17984 server_base.cc:1034] running on GCE node
I20260812 17:48:18.783156 17984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:18.783955 17984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:18.785210 17984 hybrid_clock.cc:648] HybridClock initialized: now 1786556898785173 us; error 56 us; skew 500 ppm
Aug 12 17:48:18 dist-test-slave-sb2z krb5kdc[17361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556898, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:18.788547 17984 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:48:18.789482 17984 webserver.cc:460] Webserver started at http://127.13.38.3:32799/ using document root <none> and password file <none>
I20260812 17:48:18.790086 17984 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:18.790151 17984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:18.790339 17984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:18.792665 17984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "c861fcb5433e461785ee610bc973e76e"
format_stamp: "Formatted at 2026-08-12 17:48:18 on dist-test-slave-sb2z"
server_key: "f9d4d57a7dfb35a8b040fb9173d03fc3"
server_key_iv: "f742574c85454c158b2525beae4b43a6"
server_key_version: "encryptionkey@0"
I20260812 17:48:18.793260 17984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "c861fcb5433e461785ee610bc973e76e"
format_stamp: "Formatted at 2026-08-12 17:48:18 on dist-test-slave-sb2z"
server_key: "f9d4d57a7dfb35a8b040fb9173d03fc3"
server_key_iv: "f742574c85454c158b2525beae4b43a6"
server_key_version: "encryptionkey@0"
I20260812 17:48:18.797513 17984 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:48:18.800405 18009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:18.801929 17984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:18.802086 17984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "c861fcb5433e461785ee610bc973e76e"
format_stamp: "Formatted at 2026-08-12 17:48:18 on dist-test-slave-sb2z"
server_key: "f9d4d57a7dfb35a8b040fb9173d03fc3"
server_key_iv: "f742574c85454c158b2525beae4b43a6"
server_key_version: "encryptionkey@0"
I20260812 17:48:18.802220 17984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:18.816118 17984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:18.819674 17984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:18.819842 17984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:18.820411 17984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:18.821260 17984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:18.821311 17984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:18.821353 17984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:18.821369 17984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:18.831600 17984 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:42041
I20260812 17:48:18.831629 18197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:42041 every 8 connection(s)
I20260812 17:48:18.832620 17984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:18.836917 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 17984
I20260812 17:48:18.837095 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:18.837543 13464 external_mini_cluster.cc:1428] Setting key d3feff5057d11f829a6ad1bb59fa15e9
Aug 12 17:48:18 dist-test-slave-sb2z krb5kdc[17361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556898, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:18.848136 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:18.834746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:55777 (local address 127.13.38.62:34561)
0812 17:48:18.834966 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:18.834970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:18.835997 (+ 1027us) server_negotiation.cc:407] Connection header received
0812 17:48:18.837680 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:18.837684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:18.837760 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:18.837864 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:18.839212 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.839948 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.841162 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.841401 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.841513 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:18.844143 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:18.844147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:18.844150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:18.844185 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:18.845818 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:18.846399 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:18.846408 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:18.846409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:18.846494 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:18.846860 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:18.846864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:18.846866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:18.847075 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:18.847191 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:18.847812 (+ 621us) server_negotiation.cc:299] Negotiation successful
0812 17:48:18.847963 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 17:48:18.849021 18203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:18.835174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34561 (local address 127.13.38.3:55777)
0812 17:48:18.835795 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:18.835844 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:18.837444 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:18.838063 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:18.838070 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:18.838553 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:18.839061 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:18.839075 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.840087 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.840091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:18.841021 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:18.841031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:18.841575 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:18.841579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:18.841748 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:18.842388 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:18.842405 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:18.843952 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:18.845960 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:18.845968 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:18.845985 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:18.846277 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:18.846592 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:18.846596 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:18.846598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:18.846756 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:18.847194 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:18.847205 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:18.847492 (+ 287us) client_negotiation.cc:769] Sending connection context
0812 17:48:18.847769 (+ 277us) client_negotiation.cc:240] Negotiation successful
0812 17:48:18.848128 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":143,"threads_started":1}
I20260812 17:48:18.850351 18198 heartbeater.cc:347] Connected to a master server at 127.13.38.62:34561
I20260812 17:48:18.850663 18198 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:18.851277 18198 heartbeater.cc:511] Master 127.13.38.62:34561 requested a full tablet report, sending...
I20260812 17:48:18.852569 17446 ts_manager.cc:194] Registered new tserver with Master: c861fcb5433e461785ee610bc973e76e (127.13.38.3:42041)
I20260812 17:48:18.853152 17446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:55777
I20260812 17:48:18.864233 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 17:48:18.880795 17513 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:18.879041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.13.38.62:34561)
0812 17:48:18.879268 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:18.879273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:18.879442 (+ 169us) server_negotiation.cc:407] Connection header received
0812 17:48:18.879652 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:18.879660 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:18.879709 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:18.879816 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:18.880628 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38808: BlockingRecv error: recv got EOF from 127.0.0.1:38808 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
I20260812 17:48:18.882169 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 17546
I20260812 17:48:18.899098 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 17754
I20260812 17:48:18.910848 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 17984
I20260812 17:48:18.928540 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 17384
2026-08-12T17:48:18Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2509 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:19 dist-test-slave-sb2z krb5kdc[18227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:19 dist-test-slave-sb2z krb5kdc[18227](info): set up 2 sockets
Aug 12 17:48:19 dist-test-slave-sb2z krb5kdc[18227](info): commencing operation
krb5kdc: starting...
W20260812 17:48:20.211146 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:20 dist-test-slave-sb2z krb5kdc[18227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:48:20.350782 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34731
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:34731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:20.467270 18246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:20.467679 18246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:20.467787 18246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:20.471982 18246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:20.472069 18246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:20.472105 18246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:20.472142 18246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:20.472180 18246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:20.477689 18246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:34731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:20.478896 18246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:20.480060 18246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:20.489087 18254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:20.489279 18246 server_base.cc:1034] running on GCE node
W20260812 17:48:20.489065 18257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:20.489131 18253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:20.490118 18246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:20.491266 18246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:20.492511 18246 hybrid_clock.cc:648] HybridClock initialized: now 1786556900492489 us; error 34 us; skew 500 ppm
Aug 12 17:48:20 dist-test-slave-sb2z krb5kdc[18227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556900, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:20.495815 18246 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:48:20.496850 18246 webserver.cc:460] Webserver started at http://127.13.38.62:37711/ using document root <none> and password file <none>
I20260812 17:48:20.497452 18246 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:20.497521 18246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:20.497716 18246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:20.500108 18246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "55d3f2a5a224413ba4b77541e932c320"
format_stamp: "Formatted at 2026-08-12 17:48:20 on dist-test-slave-sb2z"
server_key: "d2e03e9130c1ca212158d5e8213b83c0"
server_key_iv: "bf2130657fe138da45ad975adabdaf1b"
server_key_version: "encryptionkey@0"
I20260812 17:48:20.500756 18246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "55d3f2a5a224413ba4b77541e932c320"
format_stamp: "Formatted at 2026-08-12 17:48:20 on dist-test-slave-sb2z"
server_key: "d2e03e9130c1ca212158d5e8213b83c0"
server_key_iv: "bf2130657fe138da45ad975adabdaf1b"
server_key_version: "encryptionkey@0"
I20260812 17:48:20.505123 18246 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:48:20.507865 18267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:20.509325 18246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:48:20.509444 18246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "55d3f2a5a224413ba4b77541e932c320"
format_stamp: "Formatted at 2026-08-12 17:48:20 on dist-test-slave-sb2z"
server_key: "d2e03e9130c1ca212158d5e8213b83c0"
server_key_iv: "bf2130657fe138da45ad975adabdaf1b"
server_key_version: "encryptionkey@0"
I20260812 17:48:20.509563 18246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:20.530045 18246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:20.533628 18246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:20.533809 18246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:20.541838 18246 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:34731
I20260812 17:48:20.542214 18346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:34731 every 8 connection(s)
I20260812 17:48:20.542930 18246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:20.546041 18351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:20.548864 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 18246
I20260812 17:48:20.549010 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:20.549304 13464 external_mini_cluster.cc:1428] Setting key f8ca14bb1aebe00b0b72ffc20b11a9ea
I20260812 17:48:20.553141 18351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320: Bootstrap starting.
I20260812 17:48:20.555979 18351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:20.556926 18351 log.cc:824] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320: Log is configured to *not* fsync() on all Append() calls
Aug 12 17:48:20 dist-test-slave-sb2z krb5kdc[18227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:20.559494 18351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320: No bootstrap required, opened a new log
I20260812 17:48:20.564703 18351 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d3f2a5a224413ba4b77541e932c320" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34731 } }
I20260812 17:48:20.565034 18351 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:20.565086 18351 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55d3f2a5a224413ba4b77541e932c320, State: Initialized, Role: FOLLOWER
I20260812 17:48:20.565542 18357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:20.550776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.13.38.62:34731)
0812 17:48:20.551236 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:20.551252 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:20.551290 (+ 38us) server_negotiation.cc:407] Connection header received
0812 17:48:20.552028 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:20.552049 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:20.552402 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:20.552697 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:20.553700 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:20.554722 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:20.555982 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:20.556571 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:20.556709 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:20.559753 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:20.559760 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:20.559780 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:20.559816 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:20.561869 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:20.562384 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:20.562393 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:20.562402 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:20.562585 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:20.562931 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:20.562934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:20.562936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:20.563557 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:20.563773 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:20.564163 (+ 390us) server_negotiation.cc:299] Negotiation successful
0812 17:48:20.564501 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":145,"threads_started":1}
I20260812 17:48:20.565582 18351 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d3f2a5a224413ba4b77541e932c320" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34731 } }
I20260812 17:48:20.565732 18351 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:20.565822 18351 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:20.565927 18351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:20.567122 18351 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d3f2a5a224413ba4b77541e932c320" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34731 } }
I20260812 17:48:20.567569 18351 leader_election.cc:302] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55d3f2a5a224413ba4b77541e932c320; no voters:
I20260812 17:48:20.567963 18351 leader_election.cc:288] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:20.568140 18359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:20.568385 18359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [term 1 LEADER]: Becoming Leader. State: Replica: 55d3f2a5a224413ba4b77541e932c320, State: Running, Role: LEADER
I20260812 17:48:20.568796 18359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d3f2a5a224413ba4b77541e932c320" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34731 } }
I20260812 17:48:20.568848 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:48:20.569409 18351 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:20.570590 18361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55d3f2a5a224413ba4b77541e932c320. Latest consensus state: current_term: 1 leader_uuid: "55d3f2a5a224413ba4b77541e932c320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d3f2a5a224413ba4b77541e932c320" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34731 } } }
I20260812 17:48:20.570731 18361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:20.571089 18360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55d3f2a5a224413ba4b77541e932c320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d3f2a5a224413ba4b77541e932c320" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34731 } } }
I20260812 17:48:20.571225 18360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:20.571617 18372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:20.575058 18372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:20.582270 18372 catalog_manager.cc:1347] Generated new cluster ID: d404d8246e0946ca8088687762af0481
I20260812 17:48:20.582382 18372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:20.595938 18372 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:20.597362 18372 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:20.598162 18357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:20.591752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53128 (local address 127.13.38.62:34731)
0812 17:48:20.591920 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:20.591925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:20.592266 (+ 341us) server_negotiation.cc:407] Connection header received
0812 17:48:20.592423 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:20.592426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:20.592490 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:20.592589 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:20.593315 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:20.594170 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:20.595280 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:20.595690 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:20.595789 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:20.596919 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:20.596923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:20.596926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:20.597616 (+ 690us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:20.597655 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:20.597678 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:20.597784 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:20.597918 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 17:48:20.598011 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":60}
I20260812 17:48:20.606243 18372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55d3f2a5a224413ba4b77541e932c320: Generated new TSK 0
I20260812 17:48:20.607259 18372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:48:20.620155 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 18246
2026-08-12T17:48:20Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1694 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:20 dist-test-slave-sb2z krb5kdc[18406](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:20 dist-test-slave-sb2z krb5kdc[18406](info): set up 2 sockets
Aug 12 17:48:20 dist-test-slave-sb2z krb5kdc[18406](info): commencing operation
krb5kdc: starting...
W20260812 17:48:21.942767 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:22 dist-test-slave-sb2z krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:48:22.084002 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:35289
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:35289
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:22.206588 18429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:22.206957 18429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:22.207067 18429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:22.211391 18429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:22.211515 18429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:22.211576 18429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:22.211608 18429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:22.211652 18429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:22.217384 18429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:35289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:35289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:22.218628 18429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:22.219784 18429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:22.229278 18442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:22.229319 18440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:22.229462 18429 server_base.cc:1034] running on GCE node
W20260812 17:48:22.229278 18445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:22.230316 18429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:22.231488 18429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:22.232775 18429 hybrid_clock.cc:648] HybridClock initialized: now 1786556902232745 us; error 44 us; skew 500 ppm
Aug 12 17:48:22 dist-test-slave-sb2z krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556902, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:22.239137 18429 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:48:22.240274 18429 webserver.cc:460] Webserver started at http://127.13.38.62:45501/ using document root <none> and password file <none>
I20260812 17:48:22.240935 18429 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:22.241040 18429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:22.241307 18429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:22.243516 18429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "0baa4347ec804744b2814f52c160279b"
format_stamp: "Formatted at 2026-08-12 17:48:22 on dist-test-slave-sb2z"
server_key: "b9c5ac43c63c71a5e5dd626e8dfdcb14"
server_key_iv: "75e1b63ccb3a54f71cb56ebba36b697d"
server_key_version: "encryptionkey@0"
I20260812 17:48:22.244215 18429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "0baa4347ec804744b2814f52c160279b"
format_stamp: "Formatted at 2026-08-12 17:48:22 on dist-test-slave-sb2z"
server_key: "b9c5ac43c63c71a5e5dd626e8dfdcb14"
server_key_iv: "75e1b63ccb3a54f71cb56ebba36b697d"
server_key_version: "encryptionkey@0"
I20260812 17:48:22.248509 18429 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 17:48:22.251130 18452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:22.252511 18429 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 17:48:22.252681 18429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "0baa4347ec804744b2814f52c160279b"
format_stamp: "Formatted at 2026-08-12 17:48:22 on dist-test-slave-sb2z"
server_key: "b9c5ac43c63c71a5e5dd626e8dfdcb14"
server_key_iv: "75e1b63ccb3a54f71cb56ebba36b697d"
server_key_version: "encryptionkey@0"
I20260812 17:48:22.252810 18429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:22.274307 18429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:22.277609 18429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:22.277855 18429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:22.285425 18429 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:35289
I20260812 17:48:22.285709 18531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:35289 every 8 connection(s)
I20260812 17:48:22.286509 18429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:22.289566 18533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:22.295418 18533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b: Bootstrap starting.
I20260812 17:48:22.295784 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 18429
I20260812 17:48:22.295936 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:22.296227 13464 external_mini_cluster.cc:1428] Setting key 93ef8669ec165b8fcff74844a7d7e13e
I20260812 17:48:22.298301 18533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:22.299060 18533 log.cc:824] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:22.301117 18533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b: No bootstrap required, opened a new log
I20260812 17:48:22.305044 18533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0baa4347ec804744b2814f52c160279b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35289 } }
I20260812 17:48:22.305408 18533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 12 17:48:22 dist-test-slave-sb2z krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:22.305480 18533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0baa4347ec804744b2814f52c160279b, State: Initialized, Role: FOLLOWER
I20260812 17:48:22.306057 18533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0baa4347ec804744b2814f52c160279b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35289 } }
I20260812 17:48:22.306226 18533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:22.306318 18533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:22.306437 18533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:22.307753 18533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0baa4347ec804744b2814f52c160279b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35289 } }
I20260812 17:48:22.308137 18533 leader_election.cc:302] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0baa4347ec804744b2814f52c160279b; no voters:
I20260812 17:48:22.308509 18533 leader_election.cc:288] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:22.308650 18542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:22.308921 18542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [term 1 LEADER]: Becoming Leader. State: Replica: 0baa4347ec804744b2814f52c160279b, State: Running, Role: LEADER
I20260812 17:48:22.309269 18542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0baa4347ec804744b2814f52c160279b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35289 } }
I20260812 17:48:22.309998 18533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:22.310588 18542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0baa4347ec804744b2814f52c160279b. Latest consensus state: current_term: 1 leader_uuid: "0baa4347ec804744b2814f52c160279b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0baa4347ec804744b2814f52c160279b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35289 } } }
I20260812 17:48:22.311375 18542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:22.311259 18544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0baa4347ec804744b2814f52c160279b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0baa4347ec804744b2814f52c160279b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35289 } } }
I20260812 17:48:22.311712 18544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:22.312237 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:22.297657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53968 (local address 127.13.38.62:35289)
0812 17:48:22.298161 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:22.298178 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:22.298217 (+ 39us) server_negotiation.cc:407] Connection header received
0812 17:48:22.298924 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:22.298943 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:22.299345 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:22.299733 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:22.300538 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:22.301596 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:22.302454 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:22.302948 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:22.303152 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:22.306231 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:22.306240 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:22.306262 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:22.306304 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:22.308436 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:22.309041 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:22.309047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:22.309054 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:22.309206 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:22.309524 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:22.309528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:22.309530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:22.310112 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:22.310337 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:22.310707 (+ 370us) server_negotiation.cc:299] Negotiation successful
0812 17:48:22.311070 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":144,"threads_started":1}
I20260812 17:48:22.312825 18554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:22.315979 18554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:22.321909 18554 catalog_manager.cc:1347] Generated new cluster ID: aed33645a1484a19a7e2fc8dd151a6b4
I20260812 17:48:22.321996 18554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:22.331301 18554 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:22.332350 18554 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:22.339881 18554 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0baa4347ec804744b2814f52c160279b: Generated new TSK 0
I20260812 17:48:22.340680 18554 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:48:22.375835 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35289
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:22.495472 18574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:22.495779 18574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:22.495824 18574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:22.499809 18574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:22.499871 18574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:22.499930 18574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:22.505398 18574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:22.506515 18574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:22.507579 18574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:22.518350 18584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:22.518411 18574 server_base.cc:1034] running on GCE node
W20260812 17:48:22.518368 18583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:22.518373 18586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:22.519147 18574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:22.519909 18574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:22.521124 18574 hybrid_clock.cc:648] HybridClock initialized: now 1786556902521102 us; error 33 us; skew 500 ppm
Aug 12 17:48:22 dist-test-slave-sb2z krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556902, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:22.524353 18574 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:48:22.525421 18574 webserver.cc:460] Webserver started at http://127.13.38.1:37747/ using document root <none> and password file <none>
I20260812 17:48:22.526129 18574 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:22.526242 18574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:22.526506 18574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:22.528829 18574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "71afc46cf18f4043ad1eb4d047219876"
format_stamp: "Formatted at 2026-08-12 17:48:22 on dist-test-slave-sb2z"
server_key: "bc794f77fde5af5ad5ee9fcdc89573cc"
server_key_iv: "43f19482f5aa1cfee5447902a6bb66ee"
server_key_version: "encryptionkey@0"
I20260812 17:48:22.529523 18574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "71afc46cf18f4043ad1eb4d047219876"
format_stamp: "Formatted at 2026-08-12 17:48:22 on dist-test-slave-sb2z"
server_key: "bc794f77fde5af5ad5ee9fcdc89573cc"
server_key_iv: "43f19482f5aa1cfee5447902a6bb66ee"
server_key_version: "encryptionkey@0"
I20260812 17:48:22.533758 18574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:48:22.536613 18600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:22.538028 18574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:22.538192 18574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "71afc46cf18f4043ad1eb4d047219876"
format_stamp: "Formatted at 2026-08-12 17:48:22 on dist-test-slave-sb2z"
server_key: "bc794f77fde5af5ad5ee9fcdc89573cc"
server_key_iv: "43f19482f5aa1cfee5447902a6bb66ee"
server_key_version: "encryptionkey@0"
I20260812 17:48:22.538331 18574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:22.572867 18574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:22.576483 18574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:22.576725 18574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:22.577405 18574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:22.578317 18574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:22.578395 18574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:22.578471 18574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:22.578524 18574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:22.588713 18574 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:43961
I20260812 17:48:22.588735 18768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:43961 every 8 connection(s)
I20260812 17:48:22.589738 18574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:22.594515 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 18574
I20260812 17:48:22.594660 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:22.595010 13464 external_mini_cluster.cc:1428] Setting key 9653655dd7cf8570ffc4b5e7e2bf59e6
Aug 12 17:48:22 dist-test-slave-sb2z krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556902, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:48:22.607636 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:22.591936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:46777 (local address 127.13.38.62:35289)
0812 17:48:22.592109 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:22.592115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:22.593189 (+ 1074us) server_negotiation.cc:407] Connection header received
0812 17:48:22.594833 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:22.594837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:22.594892 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:22.595004 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:22.596573 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:22.597165 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:22.599036 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:22.599449 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:22.599608 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:22.602992 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:22.602995 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:22.602998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:22.603025 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:22.604715 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:22.605465 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:22.605469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:22.605470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:22.605560 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:22.606085 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:22.606089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:22.606090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:22.606324 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:22.606421 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:22.607280 (+ 859us) server_negotiation.cc:299] Negotiation successful
0812 17:48:22.607419 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 17:48:22.608690 18771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:22.592476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35289 (local address 127.13.38.1:46777)
0812 17:48:22.593019 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:22.593065 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:22.594524 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:22.595231 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:22.595243 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:22.595817 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:22.596390 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:22.596413 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:22.597312 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:22.597318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:22.598842 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:22.598859 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:22.599664 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:22.599667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:22.599820 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:22.600469 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:22.600487 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:22.602718 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:22.604857 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:22.604867 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:22.604892 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:22.605328 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:22.605684 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:22.605689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:22.605691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:22.605921 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:22.606472 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:22.606483 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:22.606866 (+ 383us) client_negotiation.cc:769] Sending connection context
0812 17:48:22.607191 (+ 325us) client_negotiation.cc:240] Negotiation successful
0812 17:48:22.607628 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
I20260812 17:48:22.610277 18769 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35289
I20260812 17:48:22.610580 18769 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:22.611263 18769 heartbeater.cc:511] Master 127.13.38.62:35289 requested a full tablet report, sending...
I20260812 17:48:22.613176 18474 ts_manager.cc:194] Registered new tserver with Master: 71afc46cf18f4043ad1eb4d047219876 (127.13.38.1:43961)
I20260812 17:48:22.614847 18474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:46777
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:48:22.632028 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35289
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:22.754091 18777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:22.754388 18777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:22.754436 18777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:22.758713 18777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:22.758790 18777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:22.758850 18777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:22.764494 18777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:22.765630 18777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:22.766695 18777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:22.777240 18785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:22.777333 18787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:22.777293 18784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:22.777511 18777 server_base.cc:1034] running on GCE node
I20260812 17:48:22.778067 18777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:22.778766 18777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:22.780010 18777 hybrid_clock.cc:648] HybridClock initialized: now 1786556902779986 us; error 37 us; skew 500 ppm
Aug 12 17:48:22 dist-test-slave-sb2z krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556902, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:22.783007 18777 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:48:22.783965 18777 webserver.cc:460] Webserver started at http://127.13.38.2:38279/ using document root <none> and password file <none>
I20260812 17:48:22.784513 18777 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:22.784574 18777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:22.784760 18777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:22.786888 18777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "163882d19a2842beadeaf2ea16c2240d"
format_stamp: "Formatted at 2026-08-12 17:48:22 on dist-test-slave-sb2z"
server_key: "4f3a5bdb44e0e5785f9617a17146f109"
server_key_iv: "d6c92ddba3bb783de8280b19da1db49a"
server_key_version: "encryptionkey@0"
I20260812 17:48:22.787432 18777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "163882d19a2842beadeaf2ea16c2240d"
format_stamp: "Formatted at 2026-08-12 17:48:22 on dist-test-slave-sb2z"
server_key: "4f3a5bdb44e0e5785f9617a17146f109"
server_key_iv: "d6c92ddba3bb783de8280b19da1db49a"
server_key_version: "encryptionkey@0"
I20260812 17:48:22.791651 18777 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:48:22.794476 18801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:22.796002 18777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:22.796118 18777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "163882d19a2842beadeaf2ea16c2240d"
format_stamp: "Formatted at 2026-08-12 17:48:22 on dist-test-slave-sb2z"
server_key: "4f3a5bdb44e0e5785f9617a17146f109"
server_key_iv: "d6c92ddba3bb783de8280b19da1db49a"
server_key_version: "encryptionkey@0"
I20260812 17:48:22.796216 18777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:22.808588 18777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:22.812265 18777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:22.812503 18777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:22.813235 18777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:22.814155 18777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:22.814215 18777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:22.814256 18777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:22.814277 18777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:22.823274 18777 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:34801
I20260812 17:48:22.823297 18991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:34801 every 8 connection(s)
I20260812 17:48:22.824282 18777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:22.830528 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 18777
I20260812 17:48:22.830694 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:22.831046 13464 external_mini_cluster.cc:1428] Setting key 651071f16ecacf5275bc3d8b5b6cdb23
Aug 12 17:48:22 dist-test-slave-sb2z krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556902, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:22.840101 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:22.826157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:48583 (local address 127.13.38.62:35289)
0812 17:48:22.826301 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:22.826305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:22.827268 (+ 963us) server_negotiation.cc:407] Connection header received
0812 17:48:22.828757 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:22.828760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:22.828811 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:22.828900 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:22.830187 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:22.830796 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:22.832435 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:22.832720 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:22.832806 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:22.836113 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:22.836116 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:22.836119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:22.836154 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:22.837611 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:22.838272 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:22.838276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:22.838278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:22.838367 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:22.838746 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:22.838749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:22.838751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:22.838958 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:22.839076 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:22.839711 (+ 635us) server_negotiation.cc:299] Negotiation successful
0812 17:48:22.839871 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 17:48:22.841030 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:22.826538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35289 (local address 127.13.38.2:48583)
0812 17:48:22.827100 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:22.827147 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:22.828533 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:22.829117 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:22.829124 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:22.829515 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:22.830031 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:22.830046 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:22.830948 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:22.830953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:22.832262 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:22.832273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:22.832890 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:22.832894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:22.833128 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:22.833807 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:22.833835 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:22.835927 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:22.837750 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:22.837757 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:22.837775 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:22.838156 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:22.838467 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:22.838470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:22.838472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:22.838632 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:22.839076 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:22.839084 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:22.839365 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 17:48:22.839607 (+ 242us) client_negotiation.cc:240] Negotiation successful
0812 17:48:22.839944 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":123,"threads_started":1}
I20260812 17:48:22.842315 18994 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35289
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:48:22.842617 18994 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:22.843281 18994 heartbeater.cc:511] Master 127.13.38.62:35289 requested a full tablet report, sending...
I20260812 17:48:22.844478 18474 ts_manager.cc:194] Registered new tserver with Master: 163882d19a2842beadeaf2ea16c2240d (127.13.38.2:34801)
I20260812 17:48:22.845151 18474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:48583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:48:22.863837 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35289
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:22.983917 19000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:22.984268 19000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:22.984367 19000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:22.988613 19000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:22.988724 19000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:22.988824 19000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:22.994513 19000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:22.995801 19000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:22.996930 19000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:23.008155 19016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:23.008198 19000 server_base.cc:1034] running on GCE node
W20260812 17:48:23.008119 19012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:23.008126 19009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:23.008949 19000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:23.009620 19000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:23.010874 19000 hybrid_clock.cc:648] HybridClock initialized: now 1786556903010845 us; error 40 us; skew 500 ppm
Aug 12 17:48:23 dist-test-slave-sb2z krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556903, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:23.014462 19000 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:48:23.015393 19000 webserver.cc:460] Webserver started at http://127.13.38.3:40909/ using document root <none> and password file <none>
I20260812 17:48:23.016276 19000 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:23.016371 19000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:23.016579 19000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:23.018864 19000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "0b683f42e2884b01ade3fcad9ddd9dba"
format_stamp: "Formatted at 2026-08-12 17:48:23 on dist-test-slave-sb2z"
server_key: "c098b89233fdb212801f0b4a59d8e318"
server_key_iv: "db5fa9e90297b7c0bd19b8333ff0feb2"
server_key_version: "encryptionkey@0"
I20260812 17:48:23.019667 19000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "0b683f42e2884b01ade3fcad9ddd9dba"
format_stamp: "Formatted at 2026-08-12 17:48:23 on dist-test-slave-sb2z"
server_key: "c098b89233fdb212801f0b4a59d8e318"
server_key_iv: "db5fa9e90297b7c0bd19b8333ff0feb2"
server_key_version: "encryptionkey@0"
I20260812 17:48:23.024056 19000 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:48:23.026635 19023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:23.028137 19000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:23.028263 19000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "0b683f42e2884b01ade3fcad9ddd9dba"
format_stamp: "Formatted at 2026-08-12 17:48:23 on dist-test-slave-sb2z"
server_key: "c098b89233fdb212801f0b4a59d8e318"
server_key_iv: "db5fa9e90297b7c0bd19b8333ff0feb2"
server_key_version: "encryptionkey@0"
I20260812 17:48:23.028360 19000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:23.056279 19000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:23.060066 19000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:23.060242 19000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:23.060837 19000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:23.061720 19000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:23.061770 19000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:23.061812 19000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:23.061882 19000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:23.071157 19000 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:33685
I20260812 17:48:23.071177 19225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:33685 every 8 connection(s)
I20260812 17:48:23.072216 19000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
Aug 12 17:48:23 dist-test-slave-sb2z krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556903, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:23.082342 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 19000
I20260812 17:48:23.082507 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:23.082818 13464 external_mini_cluster.cc:1428] Setting key eab292b819d79838aa35216073f2c932
I20260812 17:48:23.087353 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:23.074284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:48855 (local address 127.13.38.62:35289)
0812 17:48:23.074444 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:23.074447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:23.075312 (+ 865us) server_negotiation.cc:407] Connection header received
0812 17:48:23.076791 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:23.076794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:23.076845 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:23.076912 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:23.078198 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:23.078736 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:23.079922 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:23.080148 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:23.080235 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:23.083053 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:23.083057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:23.083060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:23.083092 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:23.085098 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:23.085771 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:23.085773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:23.085775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:23.085858 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:23.086208 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:23.086210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:23.086212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:23.086398 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:23.086479 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:23.087049 (+ 570us) server_negotiation.cc:299] Negotiation successful
0812 17:48:23.087202 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 17:48:23.088253 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:23.074658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35289 (local address 127.13.38.3:48855)
0812 17:48:23.075153 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:23.075201 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:23.076554 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:23.077101 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:23.077108 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:23.077514 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:23.078039 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:23.078054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:23.078848 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:23.078850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:23.079804 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:23.079814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:23.080305 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:23.080309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:23.080473 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:23.081153 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:23.081170 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:23.082834 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:23.085300 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:23.085307 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:23.085325 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:23.085673 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:23.085968 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:23.085971 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:23.085972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:23.086122 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:23.086508 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:23.086515 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:23.086786 (+ 271us) client_negotiation.cc:769] Sending connection context
0812 17:48:23.087003 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 17:48:23.087294 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":119,"threads_started":1}
I20260812 17:48:23.089565 19227 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35289
I20260812 17:48:23.089847 19227 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:23.090557 19227 heartbeater.cc:511] Master 127.13.38.62:35289 requested a full tablet report, sending...
I20260812 17:48:23.092011 18474 ts_manager.cc:194] Registered new tserver with Master: 0b683f42e2884b01ade3fcad9ddd9dba (127.13.38.3:33685)
I20260812 17:48:23.092653 18474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:48855
I20260812 17:48:23.097175 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 17:48:23 dist-test-slave-sb2z krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556903, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 17:48:23 dist-test-slave-sb2z krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556903, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:23.125835 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:23.117092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53980 (local address 127.13.38.62:35289)
0812 17:48:23.117289 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:23.117294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:23.117427 (+ 133us) server_negotiation.cc:407] Connection header received
0812 17:48:23.117502 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:23.117505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:23.117554 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:23.117635 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:23.118343 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:23.118870 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:23.119869 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:23.120086 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:23.120198 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:23.122764 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:23.122767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:23.122769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:23.122796 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:23.124173 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:23.124609 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:23.124611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:23.124612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:23.124688 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:23.124936 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:23.124938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:23.124939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:23.125184 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:23.125293 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:23.125531 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 17:48:23.125632 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260812 17:48:23.126291 18474 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53980
I20260812 17:48:23.127250 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 18574
I20260812 17:48:23.154386 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 18777
I20260812 17:48:23.169270 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 19000
I20260812 17:48:23.180572 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 18429
2026-08-12T17:48:23Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2559 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:23 dist-test-slave-sb2z krb5kdc[19254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:23 dist-test-slave-sb2z krb5kdc[19254](info): set up 2 sockets
Aug 12 17:48:23 dist-test-slave-sb2z krb5kdc[19254](info): commencing operation
krb5kdc: starting...
W20260812 17:48:24.460651 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:24 dist-test-slave-sb2z krb5kdc[19254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:48:24.606963 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:44081
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:44081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:24.722898 19276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:24.723196 19276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:24.723238 19276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:24.727466 19276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:24.727579 19276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:24.727604 19276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:24.727622 19276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:24.727638 19276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:24.733166 19276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:44081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:44081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:24.734450 19276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:24.735582 19276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:24.745066 19293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:24.745066 19290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:24.745294 19276 server_base.cc:1034] running on GCE node
W20260812 17:48:24.745096 19289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:24.746110 19276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:24.747308 19276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:24.748553 19276 hybrid_clock.cc:648] HybridClock initialized: now 1786556904748522 us; error 40 us; skew 500 ppm
Aug 12 17:48:24 dist-test-slave-sb2z krb5kdc[19254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556904, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:24.751950 19276 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:48:24.752969 19276 webserver.cc:460] Webserver started at http://127.13.38.62:43399/ using document root <none> and password file <none>
I20260812 17:48:24.753594 19276 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:24.753690 19276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:24.753942 19276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:24.756254 19276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "52da6aaec58c4947bca3d4a6e9964213"
format_stamp: "Formatted at 2026-08-12 17:48:24 on dist-test-slave-sb2z"
server_key: "122657459b6709201dc06a6f311fc1c9"
server_key_iv: "31ff91fcb4e03a4d383f3955fc726e1d"
server_key_version: "encryptionkey@0"
I20260812 17:48:24.756877 19276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "52da6aaec58c4947bca3d4a6e9964213"
format_stamp: "Formatted at 2026-08-12 17:48:24 on dist-test-slave-sb2z"
server_key: "122657459b6709201dc06a6f311fc1c9"
server_key_iv: "31ff91fcb4e03a4d383f3955fc726e1d"
server_key_version: "encryptionkey@0"
I20260812 17:48:24.761008 19276 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:48:24.763895 19301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:24.765372 19276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:48:24.765542 19276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "52da6aaec58c4947bca3d4a6e9964213"
format_stamp: "Formatted at 2026-08-12 17:48:24 on dist-test-slave-sb2z"
server_key: "122657459b6709201dc06a6f311fc1c9"
server_key_iv: "31ff91fcb4e03a4d383f3955fc726e1d"
server_key_version: "encryptionkey@0"
I20260812 17:48:24.765678 19276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:24.781180 19276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:24.786796 19276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:24.787034 19276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:24.798940 19276 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:44081
I20260812 17:48:24.799027 19388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:44081 every 8 connection(s)
I20260812 17:48:24.800344 19276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:24.804783 19389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:24.805575 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 19276
I20260812 17:48:24.805707 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:24.806003 13464 external_mini_cluster.cc:1428] Setting key 380c7d6fb14d230a37ea40451b35ebe3
I20260812 17:48:24.814558 19389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213: Bootstrap starting.
I20260812 17:48:24.818248 19389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213: Neither blocks nor log segments found. Creating new log.
Aug 12 17:48:24 dist-test-slave-sb2z krb5kdc[19254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:24.819451 19389 log.cc:824] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:24.822190 19389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213: No bootstrap required, opened a new log
I20260812 17:48:24.826836 19389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da6aaec58c4947bca3d4a6e9964213" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44081 } }
I20260812 17:48:24.827093 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:24.808296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.13.38.62:44081)
0812 17:48:24.808783 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:24.808807 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:24.808856 (+ 49us) server_negotiation.cc:407] Connection header received
0812 17:48:24.809933 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:24.809974 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:24.810471 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:24.810903 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:24.812003 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:24.813570 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:24.815560 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:24.816333 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:24.816517 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:24.820321 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:24.820331 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:24.820357 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:24.820401 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:24.822900 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:24.823518 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:24.823561 (+ 43us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:24.823568 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:24.823751 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:24.824115 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:24.824121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:24.824123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:24.824815 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:24.825050 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:24.825424 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 17:48:24.825826 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":185,"threads_started":1}
I20260812 17:48:24.827121 19389 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:24.827216 19389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52da6aaec58c4947bca3d4a6e9964213, State: Initialized, Role: FOLLOWER
I20260812 17:48:24.827795 19389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da6aaec58c4947bca3d4a6e9964213" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44081 } }
I20260812 17:48:24.827967 19389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:24.828035 19389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:24.828135 19389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:24.829432 19389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da6aaec58c4947bca3d4a6e9964213" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44081 } }
I20260812 17:48:24.829824 19389 leader_election.cc:302] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52da6aaec58c4947bca3d4a6e9964213; no voters:
I20260812 17:48:24.830224 19389 leader_election.cc:288] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:24.830329 19396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:24.830605 19396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [term 1 LEADER]: Becoming Leader. State: Replica: 52da6aaec58c4947bca3d4a6e9964213, State: Running, Role: LEADER
I20260812 17:48:24.830953 19396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da6aaec58c4947bca3d4a6e9964213" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44081 } }
I20260812 17:48:24.831687 19389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:24.832947 19400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52da6aaec58c4947bca3d4a6e9964213. Latest consensus state: current_term: 1 leader_uuid: "52da6aaec58c4947bca3d4a6e9964213" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da6aaec58c4947bca3d4a6e9964213" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44081 } } }
I20260812 17:48:24.833110 19400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:24.833550 19411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:24.833511 19399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52da6aaec58c4947bca3d4a6e9964213" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da6aaec58c4947bca3d4a6e9964213" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44081 } } }
I20260812 17:48:24.833604 19399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:24.836921 19411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:24.844058 19411 catalog_manager.cc:1347] Generated new cluster ID: 27f3a925de224e658d68281ea7a85c7b
I20260812 17:48:24.844164 19411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:24.859297 19411 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:24.860394 19411 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:24.865532 19411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52da6aaec58c4947bca3d4a6e9964213: Generated new TSK 0
I20260812 17:48:24.866415 19411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:48:24.903170 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44081
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:25.026602 19432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:25.026894 19432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:25.026939 19432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:25.031096 19432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:25.031160 19432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:25.031220 19432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:25.036990 19432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:25.038106 19432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:25.039166 19432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:25.049146 19441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:25.049144 19446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:25.049188 19440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:25.049582 19432 server_base.cc:1034] running on GCE node
I20260812 17:48:25.050020 19432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:25.050729 19432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:25.052047 19432 hybrid_clock.cc:648] HybridClock initialized: now 1786556905052015 us; error 39 us; skew 500 ppm
Aug 12 17:48:25 dist-test-slave-sb2z krb5kdc[19254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556905, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:25.055064 19432 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:48:25.056083 19432 webserver.cc:460] Webserver started at http://127.13.38.1:43347/ using document root <none> and password file <none>
I20260812 17:48:25.056690 19432 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:25.056787 19432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:25.057034 19432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:25.059170 19432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "d290175476ea4d37bc94304383ab0433"
format_stamp: "Formatted at 2026-08-12 17:48:25 on dist-test-slave-sb2z"
server_key: "db003cb16272b22f68928d4192f39544"
server_key_iv: "dd03b907b0a219ee94acd325e2ed76dc"
server_key_version: "encryptionkey@0"
I20260812 17:48:25.059795 19432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "d290175476ea4d37bc94304383ab0433"
format_stamp: "Formatted at 2026-08-12 17:48:25 on dist-test-slave-sb2z"
server_key: "db003cb16272b22f68928d4192f39544"
server_key_iv: "dd03b907b0a219ee94acd325e2ed76dc"
server_key_version: "encryptionkey@0"
I20260812 17:48:25.063898 19432 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 17:48:25.066622 19458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:25.068073 19432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:25.068229 19432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "d290175476ea4d37bc94304383ab0433"
format_stamp: "Formatted at 2026-08-12 17:48:25 on dist-test-slave-sb2z"
server_key: "db003cb16272b22f68928d4192f39544"
server_key_iv: "dd03b907b0a219ee94acd325e2ed76dc"
server_key_version: "encryptionkey@0"
I20260812 17:48:25.068364 19432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:25.103736 19432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:25.107079 19432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:25.107303 19432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:25.108038 19432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:25.108937 19432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:25.109014 19432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:25.109088 19432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:25.109133 19432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:25.118723 19432 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:36943
I20260812 17:48:25.118742 19650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:36943 every 8 connection(s)
I20260812 17:48:25.119747 19432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:25.121934 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 19432
I20260812 17:48:25.122058 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:25.122402 13464 external_mini_cluster.cc:1428] Setting key f12a169b4858980542b8a76bb8d9bf6e
Aug 12 17:48:25 dist-test-slave-sb2z krb5kdc[19254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556905, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:48:25.135851 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:25.121680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:50955 (local address 127.13.38.62:44081)
0812 17:48:25.121856 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:25.121860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:25.123053 (+ 1193us) server_negotiation.cc:407] Connection header received
0812 17:48:25.124717 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:25.124720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:25.124777 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:25.124863 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:25.126217 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.126753 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.128334 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.128591 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.128716 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:25.131686 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:25.131689 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:25.131691 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:25.131717 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:25.133509 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:25.134105 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:25.134109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:25.134111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:25.134201 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:25.134600 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:25.134604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:25.134605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:25.134803 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:25.134919 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:25.135478 (+ 559us) server_negotiation.cc:299] Negotiation successful
0812 17:48:25.135668 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
I20260812 17:48:25.136878 19656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:25.122213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44081 (local address 127.13.38.1:50955)
0812 17:48:25.122840 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:25.122918 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:25.124490 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:25.125069 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:25.125076 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:25.125453 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:25.126055 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:25.126079 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.126903 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.126909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:25.128177 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:25.128188 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.128764 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.128769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:25.128957 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:25.129697 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:25.129715 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:25.131465 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:25.133646 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:25.133652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:25.133676 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:25.133949 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:25.134308 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:25.134312 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:25.134314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:25.134474 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:25.134923 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:25.134932 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:25.135217 (+ 285us) client_negotiation.cc:769] Sending connection context
0812 17:48:25.135422 (+ 205us) client_negotiation.cc:240] Negotiation successful
0812 17:48:25.135798 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":148,"threads_started":1}
I20260812 17:48:25.138370 19653 heartbeater.cc:347] Connected to a master server at 127.13.38.62:44081
I20260812 17:48:25.138657 19653 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:25.139245 19653 heartbeater.cc:511] Master 127.13.38.62:44081 requested a full tablet report, sending...
I20260812 17:48:25.141139 19333 ts_manager.cc:194] Registered new tserver with Master: d290175476ea4d37bc94304383ab0433 (127.13.38.1:36943)
I20260812 17:48:25.142973 19333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:50955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:48:25.155788 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44081
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:25.276501 19661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:25.276865 19661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:25.276922 19661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:25.281256 19661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:25.281335 19661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:25.281399 19661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:25.286885 19661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:25.288151 19661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:25.289188 19661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:25.299362 19669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:25.299458 19661 server_base.cc:1034] running on GCE node
W20260812 17:48:25.299319 19673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:25.299346 19670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:25.300102 19661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:25.300753 19661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:25.301959 19661 hybrid_clock.cc:648] HybridClock initialized: now 1786556905301924 us; error 51 us; skew 500 ppm
Aug 12 17:48:25 dist-test-slave-sb2z krb5kdc[19254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556905, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:25.305104 19661 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:48:25.306102 19661 webserver.cc:460] Webserver started at http://127.13.38.2:34117/ using document root <none> and password file <none>
I20260812 17:48:25.306718 19661 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:25.306820 19661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:25.307081 19661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:25.309316 19661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "9a03110b811b4feebcdc685ec51e7605"
format_stamp: "Formatted at 2026-08-12 17:48:25 on dist-test-slave-sb2z"
server_key: "b978e1622935cb3d76add9905e462887"
server_key_iv: "c2489ea832a83db29e074f135abb28f3"
server_key_version: "encryptionkey@0"
I20260812 17:48:25.310009 19661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "9a03110b811b4feebcdc685ec51e7605"
format_stamp: "Formatted at 2026-08-12 17:48:25 on dist-test-slave-sb2z"
server_key: "b978e1622935cb3d76add9905e462887"
server_key_iv: "c2489ea832a83db29e074f135abb28f3"
server_key_version: "encryptionkey@0"
I20260812 17:48:25.314191 19661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 17:48:25.317134 19685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:25.319133 19661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 17:48:25.319283 19661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "9a03110b811b4feebcdc685ec51e7605"
format_stamp: "Formatted at 2026-08-12 17:48:25 on dist-test-slave-sb2z"
server_key: "b978e1622935cb3d76add9905e462887"
server_key_iv: "c2489ea832a83db29e074f135abb28f3"
server_key_version: "encryptionkey@0"
I20260812 17:48:25.319412 19661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:25.333873 19661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:25.337466 19661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:25.337694 19661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:25.338344 19661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:25.339222 19661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:25.339273 19661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:25.339344 19661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:25.339385 19661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:25.348722 19661 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:38935
I20260812 17:48:25.348747 19880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:38935 every 8 connection(s)
I20260812 17:48:25.349684 19661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:25.353892 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 19661
I20260812 17:48:25.354063 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:25.354353 13464 external_mini_cluster.cc:1428] Setting key 9352cb48031fe1175c87f3ba746c02ad
Aug 12 17:48:25 dist-test-slave-sb2z krb5kdc[19254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556905, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:48:25.367010 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:25.351717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:42799 (local address 127.13.38.62:44081)
0812 17:48:25.351865 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:25.351869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:25.352899 (+ 1030us) server_negotiation.cc:407] Connection header received
0812 17:48:25.354901 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:25.354905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:25.354977 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:25.355076 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:25.356711 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.357288 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.359057 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.359459 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.359666 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:25.362694 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:25.362700 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:25.362706 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:25.362745 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:25.364611 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:25.365300 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:25.365314 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:25.365315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:25.365399 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:25.365821 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:25.365825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:25.365827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:25.366025 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:25.366137 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:25.366708 (+ 571us) server_negotiation.cc:299] Negotiation successful
0812 17:48:25.366841 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 17:48:25.368129 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:25.352147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44081 (local address 127.13.38.2:42799)
0812 17:48:25.352709 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:25.352774 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:25.354499 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:25.355287 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:25.355297 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:25.355823 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:25.356515 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:25.356540 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.357446 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.357452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:25.358816 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:25.358828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.359710 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.359714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:25.359873 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:25.360589 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:25.360606 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:25.362493 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:25.364788 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:25.364795 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:25.364815 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:25.365169 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:25.365529 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:25.365532 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:25.365533 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:25.365697 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:25.366136 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:25.366145 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:25.366415 (+ 270us) client_negotiation.cc:769] Sending connection context
0812 17:48:25.366666 (+ 251us) client_negotiation.cc:240] Negotiation successful
0812 17:48:25.367033 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
I20260812 17:48:25.369450 19883 heartbeater.cc:347] Connected to a master server at 127.13.38.62:44081
I20260812 17:48:25.369724 19883 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:25.370311 19883 heartbeater.cc:511] Master 127.13.38.62:44081 requested a full tablet report, sending...
I20260812 17:48:25.371508 19333 ts_manager.cc:194] Registered new tserver with Master: 9a03110b811b4feebcdc685ec51e7605 (127.13.38.2:38935)
I20260812 17:48:25.372221 19333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:42799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:48:25.391505 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44081
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:25.513019 19890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:25.513363 19890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:25.513446 19890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:25.517673 19890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:25.517781 19890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:25.517876 19890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:25.523751 19890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:25.525065 19890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:25.526254 19890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:25.537380 19898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:25.537380 19897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:25.537468 19901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:25.537645 19890 server_base.cc:1034] running on GCE node
I20260812 17:48:25.538192 19890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:25.538947 19890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:25.540130 19890 hybrid_clock.cc:648] HybridClock initialized: now 1786556905540101 us; error 43 us; skew 500 ppm
Aug 12 17:48:25 dist-test-slave-sb2z krb5kdc[19254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556905, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:25.543363 19890 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:48:25.544397 19890 webserver.cc:460] Webserver started at http://127.13.38.3:38691/ using document root <none> and password file <none>
I20260812 17:48:25.545032 19890 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:25.545104 19890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:25.545380 19890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:25.547670 19890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "8f41d6d5697e49bb949017dc467bf858"
format_stamp: "Formatted at 2026-08-12 17:48:25 on dist-test-slave-sb2z"
server_key: "3565609352faf1cd92dff8a3929e16c1"
server_key_iv: "6b1504e5d7bc574b5ef350f7339e2bc9"
server_key_version: "encryptionkey@0"
I20260812 17:48:25.548291 19890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "8f41d6d5697e49bb949017dc467bf858"
format_stamp: "Formatted at 2026-08-12 17:48:25 on dist-test-slave-sb2z"
server_key: "3565609352faf1cd92dff8a3929e16c1"
server_key_iv: "6b1504e5d7bc574b5ef350f7339e2bc9"
server_key_version: "encryptionkey@0"
I20260812 17:48:25.552595 19890 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:48:25.555672 19909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:25.557137 19890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:48:25.557260 19890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "8f41d6d5697e49bb949017dc467bf858"
format_stamp: "Formatted at 2026-08-12 17:48:25 on dist-test-slave-sb2z"
server_key: "3565609352faf1cd92dff8a3929e16c1"
server_key_iv: "6b1504e5d7bc574b5ef350f7339e2bc9"
server_key_version: "encryptionkey@0"
I20260812 17:48:25.557438 19890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:25.585465 19890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:25.589119 19890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:25.589346 19890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:25.590011 19890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:25.590914 19890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:25.590965 19890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:25.591037 19890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:25.591079 19890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:25.600312 19890 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:39779
I20260812 17:48:25.600338 20089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:39779 every 8 connection(s)
I20260812 17:48:25.601312 19890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:25.610204 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 19890
I20260812 17:48:25.610361 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:25.610693 13464 external_mini_cluster.cc:1428] Setting key 1f4f4ab978d0dbe7b8f5d289b8b43ceb
Aug 12 17:48:25 dist-test-slave-sb2z krb5kdc[19254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556905, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:25.616294 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:25.603225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:55521 (local address 127.13.38.62:44081)
0812 17:48:25.603371 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:25.603375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:25.604359 (+ 984us) server_negotiation.cc:407] Connection header received
0812 17:48:25.605817 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:25.605820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:25.605868 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:25.605951 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:25.607199 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.607808 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.609019 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.609243 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.609339 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:25.612632 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:25.612636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:25.612638 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:25.612663 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:25.614294 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:25.614795 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:25.614797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:25.614798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:25.614880 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:25.615202 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:25.615204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:25.615205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:25.615393 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:25.615474 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:25.616067 (+ 593us) server_negotiation.cc:299] Negotiation successful
0812 17:48:25.616173 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 17:48:25.617202 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:25.603683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44081 (local address 127.13.38.3:55521)
0812 17:48:25.604189 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:25.604234 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:25.605598 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:25.606165 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:25.606172 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:25.606549 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:25.607038 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:25.607052 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.607934 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.607937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:25.608873 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:25.608882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.609412 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.609416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:25.609580 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:25.610320 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:25.610346 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:25.612439 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:25.614414 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:25.614419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:25.614433 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:25.614703 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:25.614963 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:25.614965 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:25.614966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:25.615117 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:25.615482 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:25.615488 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:25.615811 (+ 323us) client_negotiation.cc:769] Sending connection context
0812 17:48:25.615977 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 17:48:25.616292 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":122,"threads_started":1}
I20260812 17:48:25.618392 20090 heartbeater.cc:347] Connected to a master server at 127.13.38.62:44081
I20260812 17:48:25.618672 20090 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:25.619185 20090 heartbeater.cc:511] Master 127.13.38.62:44081 requested a full tablet report, sending...
I20260812 17:48:25.620316 19333 ts_manager.cc:194] Registered new tserver with Master: 8f41d6d5697e49bb949017dc467bf858 (127.13.38.3:39779)
I20260812 17:48:25.621023 19333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:55521
I20260812 17:48:25.625149 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 17:48:25 dist-test-slave-sb2z krb5kdc[19254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 17:48:25 dist-test-slave-sb2z krb5kdc[19254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.1@KRBTEST.COM
I20260812 17:48:25.657119 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:25.647093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.13.38.1:36943)
0812 17:48:25.647392 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:25.647399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:25.647425 (+ 26us) server_negotiation.cc:407] Connection header received
0812 17:48:25.647504 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:25.647509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:25.647767 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:25.647895 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:25.648652 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.649475 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.650364 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.650752 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.650849 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:25.653502 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:25.653512 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:25.653528 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:25.653555 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:25.655180 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:25.655813 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:25.655819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:25.655820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:25.655908 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:25.656167 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:25.656169 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:25.656170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:25.656436 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:25.656595 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:25.656773 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 17:48:25.656901 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":90,"threads_started":1}
Aug 12 17:48:25 dist-test-slave-sb2z krb5kdc[19254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:25.668465 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:25.660200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.13.38.62:44081)
0812 17:48:25.660357 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:25.660361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:25.660375 (+ 14us) server_negotiation.cc:407] Connection header received
0812 17:48:25.660512 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:25.660514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:25.660559 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:25.660645 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:25.661254 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.661759 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.662509 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:25.662712 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:25.662784 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:25.665305 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:25.665308 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:25.665311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:25.665337 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:25.666772 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:25.667196 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:25.667198 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:25.667200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:25.667277 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:25.667508 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:25.667510 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:25.667511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:25.667806 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:25.667914 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:25.668161 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 17:48:25.668260 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 17:48:25.668931 19333 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42242
I20260812 17:48:25.669765 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 19432
I20260812 17:48:25.690313 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 19661
I20260812 17:48:25.711516 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 19890
I20260812 17:48:25.729619 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 19276
2026-08-12T17:48:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2546 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:25 dist-test-slave-sb2z krb5kdc[20108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:25 dist-test-slave-sb2z krb5kdc[20108](info): set up 2 sockets
Aug 12 17:48:25 dist-test-slave-sb2z krb5kdc[20108](info): commencing operation
krb5kdc: starting...
W20260812 17:48:27.005348 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:27 dist-test-slave-sb2z krb5kdc[20108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:48:27.147289 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:35903
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:39289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:35903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:27.268623 20131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:27.268962 20131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:27.269032 20131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:27.273124 20131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:27.273236 20131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:27.273277 20131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:27.273319 20131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:27.273352 20131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:27.278781 20131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:35903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:35903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:27.279963 20131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:27.281054 20131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:27.289621 20143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:27.289702 20131 server_base.cc:1034] running on GCE node
W20260812 17:48:27.289652 20142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:27.289645 20146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:27.290504 20131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:27.291749 20131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:27.292927 20131 hybrid_clock.cc:648] HybridClock initialized: now 1786556907292905 us; error 43 us; skew 500 ppm
Aug 12 17:48:27 dist-test-slave-sb2z krb5kdc[20108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556907, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:27.300383 20131 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:48:27.301265 20131 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:27.301335 20131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:27.301584 20131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:27.303917 20131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "287f74d7f7d945be9fcc6f6c0b688fe6"
format_stamp: "Formatted at 2026-08-12 17:48:27 on dist-test-slave-sb2z"
server_key: "3736471028bde717b4f313e553205e60"
server_key_iv: "b8a493c40138d1d37147a54c22783c66"
server_key_version: "encryptionkey@0"
I20260812 17:48:27.304597 20131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "287f74d7f7d945be9fcc6f6c0b688fe6"
format_stamp: "Formatted at 2026-08-12 17:48:27 on dist-test-slave-sb2z"
server_key: "3736471028bde717b4f313e553205e60"
server_key_iv: "b8a493c40138d1d37147a54c22783c66"
server_key_version: "encryptionkey@0"
I20260812 17:48:27.308521 20131 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:48:27.311069 20157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:27.312422 20131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 17:48:27.312605 20131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "287f74d7f7d945be9fcc6f6c0b688fe6"
format_stamp: "Formatted at 2026-08-12 17:48:27 on dist-test-slave-sb2z"
server_key: "3736471028bde717b4f313e553205e60"
server_key_iv: "b8a493c40138d1d37147a54c22783c66"
server_key_version: "encryptionkey@0"
I20260812 17:48:27.312733 20131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:27.329780 20131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:27.333169 20131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:27.333398 20131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:27.341106 20131 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:35903
I20260812 17:48:27.341436 20242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:35903 every 8 connection(s)
I20260812 17:48:27.341871 20131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:27.344973 20243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:27.345379 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 20131
I20260812 17:48:27.345525 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:27.345809 13464 external_mini_cluster.cc:1428] Setting key 1d1c6d3a0297cd3d9ed939cf790a744a
I20260812 17:48:27.352705 20243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6: Bootstrap starting.
I20260812 17:48:27.355494 20243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6: Neither blocks nor log segments found. Creating new log.
Aug 12 17:48:27 dist-test-slave-sb2z krb5kdc[20108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:27.356447 20243 log.cc:824] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:27.358582 20243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6: No bootstrap required, opened a new log
I20260812 17:48:27.361708 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:27.347274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33310 (local address 127.13.38.62:35903)
0812 17:48:27.347714 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:27.347733 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:27.347779 (+ 46us) server_negotiation.cc:407] Connection header received
0812 17:48:27.348567 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:27.348594 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:27.349074 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:27.349476 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:27.350194 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:27.351741 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:27.352590 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:27.353246 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:27.353406 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:27.356250 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:27.356256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:27.356271 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:27.356302 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:27.358582 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:27.359162 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:27.359167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:27.359172 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:27.359292 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:27.359632 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:27.359635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:27.359636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:27.360114 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:27.360265 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:27.360595 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 17:48:27.360928 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":103,"threads_started":1}
I20260812 17:48:27.362838 20243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287f74d7f7d945be9fcc6f6c0b688fe6" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35903 } }
I20260812 17:48:27.363149 20243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:27.363210 20243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 287f74d7f7d945be9fcc6f6c0b688fe6, State: Initialized, Role: FOLLOWER
I20260812 17:48:27.363705 20243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287f74d7f7d945be9fcc6f6c0b688fe6" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35903 } }
I20260812 17:48:27.363870 20243 raft_consensus.cc:388] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:27.363986 20243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:27.364104 20243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:27.365054 20243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287f74d7f7d945be9fcc6f6c0b688fe6" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35903 } }
I20260812 17:48:27.365404 20243 leader_election.cc:302] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 287f74d7f7d945be9fcc6f6c0b688fe6; no voters:
I20260812 17:48:27.365753 20243 leader_election.cc:288] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:27.365851 20249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:27.366118 20249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [term 1 LEADER]: Becoming Leader. State: Replica: 287f74d7f7d945be9fcc6f6c0b688fe6, State: Running, Role: LEADER
I20260812 17:48:27.366508 20249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287f74d7f7d945be9fcc6f6c0b688fe6" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35903 } }
I20260812 17:48:27.367072 20243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:27.368409 20250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "287f74d7f7d945be9fcc6f6c0b688fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287f74d7f7d945be9fcc6f6c0b688fe6" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35903 } } }
I20260812 17:48:27.368567 20250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:27.369457 20252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 287f74d7f7d945be9fcc6f6c0b688fe6. Latest consensus state: current_term: 1 leader_uuid: "287f74d7f7d945be9fcc6f6c0b688fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287f74d7f7d945be9fcc6f6c0b688fe6" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35903 } } }
I20260812 17:48:27.369666 20252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6 [sys.catalog]: This master's current role is: LEADER
W20260812 17:48:27.372400 20274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:48:27.372478 20274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:48:27.372568 20262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:27.373508 20262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:27.380326 20262 catalog_manager.cc:1347] Generated new cluster ID: d421d0f7cf4a4e5c953937fbb2ba477d
I20260812 17:48:27.380421 20262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:27.401062 20262 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:27.402575 20262 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:27.410828 20262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6: Generated new TSK 0
I20260812 17:48:27.411753 20262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
W20260812 17:48:28.144906 13464 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.13.38.1: real 0.721s user 0.002s sys 0.003s
I20260812 17:48:28.145440 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35903
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:39289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:28.273710 20289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:28.274107 20289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:28.274232 20289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:28.282933 20289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:28.283022 20289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:28.283093 20289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:28.289152 20289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:28.290409 20289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:28.291594 20289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:28.302265 20300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:28.302269 20299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:28.302274 20305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:28.302333 20289 server_base.cc:1034] running on GCE node
I20260812 17:48:28.303113 20289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:28.304131 20289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:28.305335 20289 hybrid_clock.cc:648] HybridClock initialized: now 1786556908305316 us; error 30 us; skew 500 ppm
Aug 12 17:48:28 dist-test-slave-sb2z krb5kdc[20108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556908, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:28.308676 20289 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:48:28.309509 20289 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:28.309616 20289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:28.309880 20289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:28.312383 20289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "442690de23a2402bb68125345dcfdafc"
format_stamp: "Formatted at 2026-08-12 17:48:28 on dist-test-slave-sb2z"
server_key: "86a4c71365cf727c564599e45e8e299a"
server_key_iv: "082ffb04e9faa2b9b239e81a4e0186cb"
server_key_version: "encryptionkey@0"
I20260812 17:48:28.313153 20289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "442690de23a2402bb68125345dcfdafc"
format_stamp: "Formatted at 2026-08-12 17:48:28 on dist-test-slave-sb2z"
server_key: "86a4c71365cf727c564599e45e8e299a"
server_key_iv: "082ffb04e9faa2b9b239e81a4e0186cb"
server_key_version: "encryptionkey@0"
I20260812 17:48:28.317860 20289 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 17:48:28.320998 20326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:28.322607 20289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 17:48:28.322733 20289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "442690de23a2402bb68125345dcfdafc"
format_stamp: "Formatted at 2026-08-12 17:48:28 on dist-test-slave-sb2z"
server_key: "86a4c71365cf727c564599e45e8e299a"
server_key_iv: "082ffb04e9faa2b9b239e81a4e0186cb"
server_key_version: "encryptionkey@0"
I20260812 17:48:28.322840 20289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:28.338660 20289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:28.342489 20289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:28.342724 20289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:28.343408 20289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:28.344458 20289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:28.344511 20289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:28.344586 20289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:28.344619 20289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:28.355000 20289 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:34643
I20260812 17:48:28.355144 20505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:34643 every 8 connection(s)
I20260812 17:48:28.355779 20289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:28.363960 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 20289
I20260812 17:48:28.364144 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:28.364492 13464 external_mini_cluster.cc:1428] Setting key ac8eed394fe558567c6fb3ce74a403b0
Aug 12 17:48:28 dist-test-slave-sb2z krb5kdc[20108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556908, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:28.371905 20511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:28.357762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:52389 (local address 127.13.38.62:35903)
0812 17:48:28.358037 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:28.358041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:28.359192 (+ 1151us) server_negotiation.cc:407] Connection header received
0812 17:48:28.360163 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:28.360166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:28.360231 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:28.360354 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:28.361713 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:28.362274 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:28.363491 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:28.363791 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:28.363906 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:28.366809 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:28.366815 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:28.366821 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:28.366854 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:28.368954 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:28.369687 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:28.369690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:28.369692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:28.369781 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:28.370267 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:28.370270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:28.370271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:28.370522 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:28.370621 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:28.371538 (+ 917us) server_negotiation.cc:299] Negotiation successful
0812 17:48:28.371734 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":106,"threads_started":1}
I20260812 17:48:28.373230 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:28.358169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35903 (local address 127.13.38.1:52389)
0812 17:48:28.358995 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:28.359050 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:28.359933 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:28.360585 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:28.360614 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:28.361036 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:28.361553 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:28.361567 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:28.362416 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:28.362420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:28.363345 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:28.363356 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:28.363972 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:28.363977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:28.364210 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:28.364919 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:28.364937 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:28.366589 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:28.369094 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:28.369104 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:28.369126 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:28.369545 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:28.369886 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:28.369890 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:28.369895 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:28.370118 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:28.370739 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:28.370750 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:28.371157 (+ 407us) client_negotiation.cc:769] Sending connection context
0812 17:48:28.371411 (+ 254us) client_negotiation.cc:240] Negotiation successful
0812 17:48:28.371855 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":113,"threads_started":1}
I20260812 17:48:28.374827 20508 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35903
I20260812 17:48:28.375161 20508 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:28.375856 20508 heartbeater.cc:511] Master 127.13.38.62:35903 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:48:28.377919 20186 ts_manager.cc:194] Registered new tserver with Master: 442690de23a2402bb68125345dcfdafc (127.13.38.1:34643)
I20260812 17:48:28.379137 20186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:52389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
W20260812 17:48:29.137040 13464 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.13.38.2: real 0.771s user 0.000s sys 0.004s
I20260812 17:48:29.137820 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35903
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:39289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:29.263167 20517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:29.263501 20517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:29.263629 20517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:29.267844 20517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:29.267936 20517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:29.268057 20517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:29.273700 20517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:29.274907 20517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:29.276110 20517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 17:48:29.286592 20517 server_base.cc:1034] running on GCE node
W20260812 17:48:29.286612 20523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:29.286602 20527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:29.286602 20522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:29.287442 20517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:29.288234 20517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:29.289417 20517 hybrid_clock.cc:648] HybridClock initialized: now 1786556909289398 us; error 33 us; skew 500 ppm
Aug 12 17:48:29 dist-test-slave-sb2z krb5kdc[20108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556909, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:29.292622 20517 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:48:29.293525 20517 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:29.293596 20517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:29.293829 20517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:29.296111 20517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "2966a0d824594830bffc6e7bb8fe8df5"
format_stamp: "Formatted at 2026-08-12 17:48:29 on dist-test-slave-sb2z"
server_key: "4b1b7e14b27716d16b6ad6d42ca8100e"
server_key_iv: "ac9e742af3a32d7bd47e1f18f940f79d"
server_key_version: "encryptionkey@0"
I20260812 17:48:29.296700 20517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "2966a0d824594830bffc6e7bb8fe8df5"
format_stamp: "Formatted at 2026-08-12 17:48:29 on dist-test-slave-sb2z"
server_key: "4b1b7e14b27716d16b6ad6d42ca8100e"
server_key_iv: "ac9e742af3a32d7bd47e1f18f940f79d"
server_key_version: "encryptionkey@0"
I20260812 17:48:29.300637 20517 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:48:29.303478 20537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:29.304854 20517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:48:29.304965 20517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "2966a0d824594830bffc6e7bb8fe8df5"
format_stamp: "Formatted at 2026-08-12 17:48:29 on dist-test-slave-sb2z"
server_key: "4b1b7e14b27716d16b6ad6d42ca8100e"
server_key_iv: "ac9e742af3a32d7bd47e1f18f940f79d"
server_key_version: "encryptionkey@0"
I20260812 17:48:29.305133 20517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:29.316031 20517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:29.319819 20517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:29.320036 20517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:29.320716 20517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:29.321820 20517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:29.321887 20517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:29.321961 20517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:29.322036 20517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260812 17:48:29.332731 20517 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:43321
I20260812 17:48:29.332749 20727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:43321 every 8 connection(s)
I20260812 17:48:29.333385 20517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:29.336359 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 20517
I20260812 17:48:29.336519 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:29.336890 13464 external_mini_cluster.cc:1428] Setting key 6131543e985d3cfb4140fcfe06823a24
Aug 12 17:48:29 dist-test-slave-sb2z krb5kdc[20108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556909, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:29.350183 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.335179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:36223 (local address 127.13.38.62:35903)
0812 17:48:29.335461 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:29.335465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:29.336849 (+ 1384us) server_negotiation.cc:407] Connection header received
0812 17:48:29.338034 (+ 1185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:29.338038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:29.338087 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:29.338189 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:29.339967 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.340522 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.342229 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.342455 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.342534 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.345802 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:29.345806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:29.345809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:29.345840 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:29.347677 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:29.348379 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:29.348382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:29.348383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:29.348469 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:29.348914 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:29.348916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:29.348917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:29.349112 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:29.349195 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:29.349901 (+ 706us) server_negotiation.cc:299] Negotiation successful
0812 17:48:29.350012 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:48:29.351320 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.335630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35903 (local address 127.13.38.2:36223)
0812 17:48:29.336660 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:29.336728 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:29.337767 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:29.338405 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:29.338414 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:29.338944 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:29.339742 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.339767 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.340671 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.340678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.342103 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.342116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.342603 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.342607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.342796 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.343416 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:29.343437 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:29.345583 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:29.347805 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:29.347815 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:29.347837 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:29.348248 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:29.348572 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:29.348578 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:29.348580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:29.348808 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:29.349237 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:29.349245 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:29.349615 (+ 370us) client_negotiation.cc:769] Sending connection context
0812 17:48:29.349857 (+ 242us) client_negotiation.cc:240] Negotiation successful
0812 17:48:29.350284 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":100,"threads_started":1}
I20260812 17:48:29.352663 20728 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35903
I20260812 17:48:29.352912 20728 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:29.353487 20728 heartbeater.cc:511] Master 127.13.38.62:35903 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
I20260812 17:48:29.354630 20186 ts_manager.cc:194] Registered new tserver with Master: 2966a0d824594830bffc6e7bb8fe8df5 (127.13.38.2:43321)
I20260812 17:48:29.355249 20186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:36223
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:48:29.376158 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35903
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:39289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:48:29.382536 20508 heartbeater.cc:503] Master 127.13.38.62:35903 was elected leader, sending a full tablet report...
W20260812 17:48:29.496899 20736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:29.497275 20736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:29.497376 20736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:29.501688 20736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:29.501788 20736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:29.501888 20736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:29.507637 20736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:29.508968 20736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:29.510272 20736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:29.520386 20747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:29.520457 20742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:29.520650 20743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:29.520483 20736 server_base.cc:1034] running on GCE node
I20260812 17:48:29.521203 20736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:29.521852 20736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:29.523063 20736 hybrid_clock.cc:648] HybridClock initialized: now 1786556909523044 us; error 31 us; skew 500 ppm
Aug 12 17:48:29 dist-test-slave-sb2z krb5kdc[20108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556909, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:29.526221 20736 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:48:29.527094 20736 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:29.527164 20736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:29.527343 20736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:29.530402 20736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "7c6686bb814044829df8f0f7350409b5"
format_stamp: "Formatted at 2026-08-12 17:48:29 on dist-test-slave-sb2z"
server_key: "ed28bdd3d9d4d4a7a777b83710caeb4d"
server_key_iv: "cd853db5c564159501deff4339c2adba"
server_key_version: "encryptionkey@0"
I20260812 17:48:29.531046 20736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "7c6686bb814044829df8f0f7350409b5"
format_stamp: "Formatted at 2026-08-12 17:48:29 on dist-test-slave-sb2z"
server_key: "ed28bdd3d9d4d4a7a777b83710caeb4d"
server_key_iv: "cd853db5c564159501deff4339c2adba"
server_key_version: "encryptionkey@0"
I20260812 17:48:29.534996 20736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:48:29.537663 20754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:29.539052 20736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:29.539191 20736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "7c6686bb814044829df8f0f7350409b5"
format_stamp: "Formatted at 2026-08-12 17:48:29 on dist-test-slave-sb2z"
server_key: "ed28bdd3d9d4d4a7a777b83710caeb4d"
server_key_iv: "cd853db5c564159501deff4339c2adba"
server_key_version: "encryptionkey@0"
I20260812 17:48:29.539310 20736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:29.551472 20736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:29.554968 20736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:29.555193 20736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:29.555841 20736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:29.556712 20736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:29.556762 20736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:29.556830 20736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:29.556864 20736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:29.566370 20736 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:42721
I20260812 17:48:29.566387 20936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:42721 every 8 connection(s)
I20260812 17:48:29.567068 20736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:29.574375 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 20736
I20260812 17:48:29.574527 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:29.574842 13464 external_mini_cluster.cc:1428] Setting key c70297f9f3fefe8d8d5d921d3ae0c167
Aug 12 17:48:29 dist-test-slave-sb2z krb5kdc[20108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556909, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:29.582583 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.569084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:45731 (local address 127.13.38.62:35903)
0812 17:48:29.569238 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:29.569241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:29.570598 (+ 1357us) server_negotiation.cc:407] Connection header received
0812 17:48:29.571578 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:29.571582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:29.571644 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:29.571761 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:29.573036 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.573585 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.575220 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.575619 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.575750 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.578814 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:29.578818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:29.578820 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:29.578850 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:29.580295 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:29.580854 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:29.580857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:29.580858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:29.580953 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:29.581381 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:29.581383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:29.581384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:29.581588 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:29.581662 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:29.582292 (+ 630us) server_negotiation.cc:299] Negotiation successful
0812 17:48:29.582428 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 17:48:29.583639 20940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.569534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35903 (local address 127.13.38.3:45731)
0812 17:48:29.570410 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:29.570460 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:29.571309 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:29.571950 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:29.571957 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:29.572366 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:29.572873 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.572887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.573742 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.573750 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.575057 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.575070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.575796 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.575800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.576085 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.576822 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:29.576844 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:29.578643 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:29.580419 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:29.580425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:29.580440 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:29.580755 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:29.581098 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:29.581101 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:29.581103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:29.581284 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:29.581731 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:29.581737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:29.582021 (+ 284us) client_negotiation.cc:769] Sending connection context
0812 17:48:29.582206 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 17:48:29.582503 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":119,"threads_started":1}
I20260812 17:48:29.584997 20937 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35903
I20260812 17:48:29.585287 20937 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:29.585803 20937 heartbeater.cc:511] Master 127.13.38.62:35903 requested a full tablet report, sending...
I20260812 17:48:29.587021 20186 ts_manager.cc:194] Registered new tserver with Master: 7c6686bb814044829df8f0f7350409b5 (127.13.38.3:42721)
I20260812 17:48:29.587733 20186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:45731
I20260812 17:48:29.589670 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:48:29.599481 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.591689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33320 (local address 127.13.38.62:35903)
0812 17:48:29.591852 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:29.591858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:29.592096 (+ 238us) server_negotiation.cc:407] Connection header received
0812 17:48:29.592164 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:29.592166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:29.592208 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:29.592282 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:29.593013 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.593536 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.594341 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.594551 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.594627 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.596093 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:29.596096 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:29.596099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:29.596122 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:29.597660 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:29.598127 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:29.598130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:29.598131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:29.598210 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:29.598527 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:29.598529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:29.598530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:29.598765 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:29.598874 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:29.599144 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 17:48:29.599251 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 17:48:29.602766 20186 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:48:29.605851 20186 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:29.621833 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.615334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:43321 (local address 127.0.0.1:51594)
0812 17:48:29.616879 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:29.616892 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:29.617031 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:29.617597 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:29.617600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:29.617619 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:29.617717 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.617725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.618795 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.618797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.620365 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.620378 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.621459 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.621465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.621549 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.621572 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 17:48:29.621645 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 17:48:29.621697 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1471,"mutex_wait_us":274,"thread_start_us":116,"threads_started":1}
I20260812 17:48:29.621833 20958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.615026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:42721 (local address 127.0.0.1:34688)
0812 17:48:29.616563 (+ 1537us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:29.616577 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:29.616660 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:29.617170 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:29.617174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:29.617193 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:29.617290 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.617300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.618744 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.618748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.620102 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.620114 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.621223 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.621226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.621322 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.621356 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 17:48:29.621501 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 17:48:29.621653 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1475,"mutex_wait_us":207,"thread_start_us":130,"threads_started":1}
I20260812 17:48:29.622367 20961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.615771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34688 (local address 127.13.38.3:42721)
0812 17:48:29.616204 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:29.616215 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:29.616670 (+ 455us) server_negotiation.cc:407] Connection header received
0812 17:48:29.616792 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:29.616800 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:29.617003 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:29.617184 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:29.617429 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.618576 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.620238 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.621086 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.621210 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.621378 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:29.621555 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 17:48:29.621672 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":94,"threads_started":1}
I20260812 17:48:29.622561 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.615850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51594 (local address 127.13.38.2:43321)
0812 17:48:29.616436 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:29.616445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:29.616969 (+ 524us) server_negotiation.cc:407] Connection header received
0812 17:48:29.617198 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:29.617204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:29.617455 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:29.617614 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:29.617844 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.618683 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.620521 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.621320 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.621436 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.621590 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:29.622262 (+ 672us) server_negotiation.cc:299] Negotiation successful
0812 17:48:29.622383 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":125,"threads_started":1}
I20260812 17:48:29.624203 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.614860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:34643 (local address 127.0.0.1:42964)
0812 17:48:29.615814 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:29.615872 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:29.616011 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:29.616901 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:29.616910 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:29.616941 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:29.617099 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.617109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.618744 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.618748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.620375 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.620381 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.623790 (+ 3409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.623794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.623883 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.623908 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 17:48:29.623997 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 17:48:29.624071 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":107,"thread_start_us":141,"threads_started":1}
I20260812 17:48:29.624221 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.615034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42964 (local address 127.13.38.1:34643)
0812 17:48:29.616344 (+ 1310us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:29.616349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:29.616371 (+ 22us) server_negotiation.cc:407] Connection header received
0812 17:48:29.616439 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:29.616443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:29.616732 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:29.616891 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:29.617238 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.618604 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.621671 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.623659 (+ 1988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.623751 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.623863 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:29.624040 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 17:48:29.624115 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1193,"thread_start_us":113,"threads_started":1}
I20260812 17:48:29.625746 20833 tablet_service.cc:1467] Processing CreateTablet for tablet 44e005063be447848742712c9643140b (DEFAULT_TABLE table=test-table [id=9c6af668f5924915a559078720276880]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:29.627085 20833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e005063be447848742712c9643140b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:29.627732 20399 tablet_service.cc:1467] Processing CreateTablet for tablet 44e005063be447848742712c9643140b (DEFAULT_TABLE table=test-table [id=9c6af668f5924915a559078720276880]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:29.628842 20399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e005063be447848742712c9643140b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:29.635151 20965 tablet_bootstrap.cc:492] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5: Bootstrap starting.
I20260812 17:48:29.637979 20965 tablet_bootstrap.cc:654] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:29.637874 20622 tablet_service.cc:1467] Processing CreateTablet for tablet 44e005063be447848742712c9643140b (DEFAULT_TABLE table=test-table [id=9c6af668f5924915a559078720276880]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:29.638695 20969 tablet_bootstrap.cc:492] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc: Bootstrap starting.
I20260812 17:48:29.639217 20622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e005063be447848742712c9643140b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:29.639322 20965 log.cc:824] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:29.640995 20969 tablet_bootstrap.cc:654] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:29.641708 20965 tablet_bootstrap.cc:492] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5: No bootstrap required, opened a new log
I20260812 17:48:29.641746 20969 log.cc:824] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:29.641937 20965 ts_tablet_manager.cc:1397] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 17:48:29.644034 20969 tablet_bootstrap.cc:492] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc: No bootstrap required, opened a new log
I20260812 17:48:29.644289 20969 ts_tablet_manager.cc:1397] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 17:48:29.646222 20965 raft_consensus.cc:348] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442690de23a2402bb68125345dcfdafc" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34643 } } peers { permanent_uuid: "7c6686bb814044829df8f0f7350409b5" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42721 } } peers { permanent_uuid: "2966a0d824594830bffc6e7bb8fe8df5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43321 } }
I20260812 17:48:29.646554 20965 raft_consensus.cc:374] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:29.646618 20965 raft_consensus.cc:729] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c6686bb814044829df8f0f7350409b5, State: Initialized, Role: FOLLOWER
I20260812 17:48:29.647189 20972 tablet_bootstrap.cc:492] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5: Bootstrap starting.
I20260812 17:48:29.647153 20965 consensus_queue.cc:260] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442690de23a2402bb68125345dcfdafc" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34643 } } peers { permanent_uuid: "7c6686bb814044829df8f0f7350409b5" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42721 } } peers { permanent_uuid: "2966a0d824594830bffc6e7bb8fe8df5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43321 } }
I20260812 17:48:29.648265 20965 ts_tablet_manager.cc:1428] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260812 17:48:29.648379 20937 heartbeater.cc:503] Master 127.13.38.62:35903 was elected leader, sending a full tablet report...
I20260812 17:48:29.649855 20972 tablet_bootstrap.cc:654] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:29.649866 20969 raft_consensus.cc:348] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442690de23a2402bb68125345dcfdafc" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34643 } } peers { permanent_uuid: "7c6686bb814044829df8f0f7350409b5" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42721 } } peers { permanent_uuid: "2966a0d824594830bffc6e7bb8fe8df5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43321 } }
I20260812 17:48:29.650163 20969 raft_consensus.cc:374] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:29.650249 20969 raft_consensus.cc:729] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442690de23a2402bb68125345dcfdafc, State: Initialized, Role: FOLLOWER
I20260812 17:48:29.650753 20969 consensus_queue.cc:260] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442690de23a2402bb68125345dcfdafc" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34643 } } peers { permanent_uuid: "7c6686bb814044829df8f0f7350409b5" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42721 } } peers { permanent_uuid: "2966a0d824594830bffc6e7bb8fe8df5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43321 } }
I20260812 17:48:29.651167 20972 log.cc:824] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:29.651921 20969 ts_tablet_manager.cc:1428] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 17:48:29.653746 20972 tablet_bootstrap.cc:492] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5: No bootstrap required, opened a new log
I20260812 17:48:29.653970 20972 ts_tablet_manager.cc:1397] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 17:48:29.657250 20972 raft_consensus.cc:348] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442690de23a2402bb68125345dcfdafc" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34643 } } peers { permanent_uuid: "7c6686bb814044829df8f0f7350409b5" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42721 } } peers { permanent_uuid: "2966a0d824594830bffc6e7bb8fe8df5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43321 } }
I20260812 17:48:29.657482 20972 raft_consensus.cc:374] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:29.657554 20972 raft_consensus.cc:729] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2966a0d824594830bffc6e7bb8fe8df5, State: Initialized, Role: FOLLOWER
I20260812 17:48:29.657976 20972 consensus_queue.cc:260] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442690de23a2402bb68125345dcfdafc" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34643 } } peers { permanent_uuid: "7c6686bb814044829df8f0f7350409b5" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42721 } } peers { permanent_uuid: "2966a0d824594830bffc6e7bb8fe8df5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43321 } }
I20260812 17:48:29.658684 20728 heartbeater.cc:503] Master 127.13.38.62:35903 was elected leader, sending a full tablet report...
I20260812 17:48:29.658886 20972 ts_tablet_manager.cc:1428] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260812 17:48:29.716729 20973 raft_consensus.cc:482] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:29.716964 20973 raft_consensus.cc:504] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442690de23a2402bb68125345dcfdafc" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34643 } } peers { permanent_uuid: "7c6686bb814044829df8f0f7350409b5" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42721 } } peers { permanent_uuid: "2966a0d824594830bffc6e7bb8fe8df5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43321 } }
I20260812 17:48:29.718243 20973 leader_election.cc:288] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442690de23a2402bb68125345dcfdafc (127.13.38.1:34643), 2966a0d824594830bffc6e7bb8fe8df5 (127.13.38.2:43321)
I20260812 17:48:29.722043 20940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.718507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:43321 (local address 127.13.38.3:57807)
0812 17:48:29.718654 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:29.718673 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:29.718778 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:29.719076 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:29.719079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:29.719097 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:29.719225 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.719231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.720099 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.720102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.720913 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.720921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.721588 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.721592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.721701 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.721730 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 17:48:29.721823 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 17:48:29.721900 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 17:48:29.722043 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.718694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:57807 (local address 127.13.38.2:43321)
0812 17:48:29.718831 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:29.718836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:29.718853 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:48:29.718916 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:29.718918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:29.718960 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:29.719055 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:29.719348 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.719939 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.721047 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.721455 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.721593 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.721664 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:29.721843 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 17:48:29.721916 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:48:29.722553 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.718602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:36401 (local address 127.13.38.1:34643)
0812 17:48:29.718733 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:29.718738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:29.719139 (+ 401us) server_negotiation.cc:407] Connection header received
0812 17:48:29.719451 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:29.719455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:29.719554 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:29.719658 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:29.719953 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.720447 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.721506 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.721919 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.722051 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.722119 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:29.722332 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 17:48:29.722414 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 17:48:29.722905 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.718507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:34643 (local address 127.13.38.3:36401)
0812 17:48:29.719033 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:29.719045 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:29.719183 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:29.719684 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:29.719687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:29.719704 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:29.719837 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.719842 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.720578 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.720582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.721372 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:29.721380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.722072 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.722076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:29.722178 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.722204 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 17:48:29.722729 (+ 525us) client_negotiation.cc:240] Negotiation successful
0812 17:48:29.722799 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":129,"thread_start_us":185,"threads_started":1}
I20260812 17:48:29.722837 20654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e005063be447848742712c9643140b" candidate_uuid: "7c6686bb814044829df8f0f7350409b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2966a0d824594830bffc6e7bb8fe8df5" is_pre_election: true
I20260812 17:48:29.723253 20654 raft_consensus.cc:2454] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c6686bb814044829df8f0f7350409b5 in term 0.
I20260812 17:48:29.723694 20430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e005063be447848742712c9643140b" candidate_uuid: "7c6686bb814044829df8f0f7350409b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442690de23a2402bb68125345dcfdafc" is_pre_election: true
I20260812 17:48:29.723790 20762 leader_election.cc:302] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2966a0d824594830bffc6e7bb8fe8df5, 7c6686bb814044829df8f0f7350409b5; no voters:
I20260812 17:48:29.724077 20430 raft_consensus.cc:2454] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c6686bb814044829df8f0f7350409b5 in term 0.
I20260812 17:48:29.724118 20973 raft_consensus.cc:2781] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:29.724217 20973 raft_consensus.cc:482] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:29.724272 20973 raft_consensus.cc:3037] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:29.725432 20973 raft_consensus.cc:504] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442690de23a2402bb68125345dcfdafc" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34643 } } peers { permanent_uuid: "7c6686bb814044829df8f0f7350409b5" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42721 } } peers { permanent_uuid: "2966a0d824594830bffc6e7bb8fe8df5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43321 } }
I20260812 17:48:29.725839 20973 leader_election.cc:288] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [CANDIDATE]: Term 1 election: Requested vote from peers 442690de23a2402bb68125345dcfdafc (127.13.38.1:34643), 2966a0d824594830bffc6e7bb8fe8df5 (127.13.38.2:43321)
I20260812 17:48:29.726255 20654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e005063be447848742712c9643140b" candidate_uuid: "7c6686bb814044829df8f0f7350409b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2966a0d824594830bffc6e7bb8fe8df5"
I20260812 17:48:29.726267 20430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e005063be447848742712c9643140b" candidate_uuid: "7c6686bb814044829df8f0f7350409b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442690de23a2402bb68125345dcfdafc"
I20260812 17:48:29.726406 20654 raft_consensus.cc:3037] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:29.726447 20430 raft_consensus.cc:3037] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:29.727770 20430 raft_consensus.cc:2454] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c6686bb814044829df8f0f7350409b5 in term 1.
I20260812 17:48:29.727773 20654 raft_consensus.cc:2454] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c6686bb814044829df8f0f7350409b5 in term 1.
I20260812 17:48:29.728191 20755 leader_election.cc:302] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2966a0d824594830bffc6e7bb8fe8df5, 442690de23a2402bb68125345dcfdafc, 7c6686bb814044829df8f0f7350409b5; no voters:
I20260812 17:48:29.728394 20973 raft_consensus.cc:2781] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:29.728616 20973 raft_consensus.cc:686] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 1 LEADER]: Becoming Leader. State: Replica: 7c6686bb814044829df8f0f7350409b5, State: Running, Role: LEADER
I20260812 17:48:29.728976 20973 consensus_queue.cc:237] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442690de23a2402bb68125345dcfdafc" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34643 } } peers { permanent_uuid: "7c6686bb814044829df8f0f7350409b5" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42721 } } peers { permanent_uuid: "2966a0d824594830bffc6e7bb8fe8df5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43321 } }
I20260812 17:48:29.732529 20184 catalog_manager.cc:5495] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c6686bb814044829df8f0f7350409b5 (127.13.38.3). New cstate: current_term: 1 leader_uuid: "7c6686bb814044829df8f0f7350409b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "442690de23a2402bb68125345dcfdafc" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c6686bb814044829df8f0f7350409b5" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2966a0d824594830bffc6e7bb8fe8df5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43321 } health_report { overall_health: UNKNOWN } } }
I20260812 17:48:29.746871 20961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.742987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.13.38.3:42721)
0812 17:48:29.743117 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:29.743122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:29.743137 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:48:29.743317 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:29.743320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:29.743378 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:29.743474 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:29.743901 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.744497 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.745446 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:29.745657 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:29.745741 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:29.746083 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:29.746518 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:29.746612 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:29.746703 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 17:48:29.746754 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 17:48:29.753424 20654 raft_consensus.cc:1261] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5 [term 1 FOLLOWER]: Refusing update from remote peer 7c6686bb814044829df8f0f7350409b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:29.753424 20430 raft_consensus.cc:1261] T 44e005063be447848742712c9643140b P 442690de23a2402bb68125345dcfdafc [term 1 FOLLOWER]: Refusing update from remote peer 7c6686bb814044829df8f0f7350409b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:29.754251 20979 consensus_queue.cc:1035] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2966a0d824594830bffc6e7bb8fe8df5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:29.754446 20973 consensus_queue.cc:1035] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442690de23a2402bb68125345dcfdafc" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:29.772614 20184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:29.772841 20184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:29.775755 20184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 287f74d7f7d945be9fcc6f6c0b688fe6: Sending DeleteTablet for 3 replicas of tablet 44e005063be447848742712c9643140b
I20260812 17:48:29.776767 20399 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e005063be447848742712c9643140b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:29 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:42964
I20260812 17:48:29.776769 20833 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e005063be447848742712c9643140b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:29 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:34688
I20260812 17:48:29.776908 20622 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e005063be447848742712c9643140b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:29 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:51594
I20260812 17:48:29.777463 21002 tablet_replica.cc:330] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5: stopping tablet replica
I20260812 17:48:29.777624 21000 tablet_replica.cc:330] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5: stopping tablet replica
I20260812 17:48:29.777850 21002 raft_consensus.cc:2229] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:29.778081 21000 raft_consensus.cc:2229] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:29.778179 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 20289
I20260812 17:48:29.778227 21002 raft_consensus.cc:2258] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:29.778739 21000 raft_consensus.cc:2258] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:29.784294 21000 ts_tablet_manager.cc:1905] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:29.784386 21002 ts_tablet_manager.cc:1905] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:29.786666 21002 ts_tablet_manager.cc:1918] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:29.786806 21002 log.cc:1196] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/44e005063be447848742712c9643140b
I20260812 17:48:29.787173 21002 ts_tablet_manager.cc:1939] T 44e005063be447848742712c9643140b P 2966a0d824594830bffc6e7bb8fe8df5: Deleting consensus metadata
I20260812 17:48:29.788094 21000 ts_tablet_manager.cc:1918] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:29.788216 21000 log.cc:1196] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/44e005063be447848742712c9643140b
I20260812 17:48:29.788623 21000 ts_tablet_manager.cc:1939] T 44e005063be447848742712c9643140b P 7c6686bb814044829df8f0f7350409b5: Deleting consensus metadata
I20260812 17:48:29.788664 20162 catalog_manager.cc:4841] TS 2966a0d824594830bffc6e7bb8fe8df5 (127.13.38.2:43321): tablet 44e005063be447848742712c9643140b (table test-table [id=9c6af668f5924915a559078720276880]) successfully deleted
I20260812 17:48:29.789971 20160 catalog_manager.cc:4841] TS 7c6686bb814044829df8f0f7350409b5 (127.13.38.3:42721): tablet 44e005063be447848742712c9643140b (table test-table [id=9c6af668f5924915a559078720276880]) successfully deleted
W20260812 17:48:29.792798 20158 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:34643 (error 108)
I20260812 17:48:29.793826 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 20517
I20260812 17:48:29.794047 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:29.793677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:34643 (local address 127.0.0.1:42966)
0812 17:48:29.793839 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:29.793933 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 17:48:29.801086 20158 catalog_manager.cc:4584] TS 442690de23a2402bb68125345dcfdafc (127.13.38.1:34643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44e005063be447848742712c9643140b: Network error: Client connection negotiation failed: client connection to 127.13.38.1:34643: connect: Connection refused (error 111)
I20260812 17:48:29.809814 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 20736
I20260812 17:48:29.827292 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 20131
2026-08-12T17:48:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4099 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T17:48:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:29Z Disabled control of system clock
I20260812 17:48:29.869222 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:40407
--webserver_interface=127.13.38.62
--webserver_port=0
--builtin_ntp_servers=127.13.38.20:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:40407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 17:48:29.989203 21010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:29.989537 21010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:29.989579 21010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:29.994177 21010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:29.994277 21010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:29.994294 21010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:29.994313 21010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:29.994328 21010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:29.999847 21010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:40407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:40407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:30.001366 21010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:30.003049 21010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:30.012565 21020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:30.012598 21023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:30.012581 21019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:30.012862 21010 server_base.cc:1034] running on GCE node
I20260812 17:48:30.013633 21010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:30.014894 21010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:30.016099 21010 hybrid_clock.cc:648] HybridClock initialized: now 1786556910016069 us; error 44 us; skew 500 ppm
I20260812 17:48:30.018265 21010 webserver.cc:460] Webserver started at http://127.13.38.62:32771/ using document root <none> and password file <none>
I20260812 17:48:30.018882 21010 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:30.018949 21010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:30.019146 21010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:30.021565 21010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "0ebf5bae09c149598dab7c8401c7f8ee"
format_stamp: "Formatted at 2026-08-12 17:48:30 on dist-test-slave-sb2z"
server_key: "cc4174266de6805289c3cec380fe7023"
server_key_iv: "197a44afdc3d2f19812b2374ce7f495f"
server_key_version: "encryptionkey@0"
I20260812 17:48:30.022150 21010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "0ebf5bae09c149598dab7c8401c7f8ee"
format_stamp: "Formatted at 2026-08-12 17:48:30 on dist-test-slave-sb2z"
server_key: "cc4174266de6805289c3cec380fe7023"
server_key_iv: "197a44afdc3d2f19812b2374ce7f495f"
server_key_version: "encryptionkey@0"
I20260812 17:48:30.026377 21010 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 17:48:30.029215 21037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:30.030652 21010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 17:48:30.030813 21010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "0ebf5bae09c149598dab7c8401c7f8ee"
format_stamp: "Formatted at 2026-08-12 17:48:30 on dist-test-slave-sb2z"
server_key: "cc4174266de6805289c3cec380fe7023"
server_key_iv: "197a44afdc3d2f19812b2374ce7f495f"
server_key_version: "encryptionkey@0"
I20260812 17:48:30.030952 21010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:30.036064 21010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:30.036799 21010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:30.036966 21010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:30.044605 21010 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:40407
I20260812 17:48:30.044654 21115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:40407 every 8 connection(s)
I20260812 17:48:30.045655 21010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:30.046032 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 21010
I20260812 17:48:30.046190 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:30.046591 13464 external_mini_cluster.cc:1428] Setting key e66b5e0c47ccaa78a3e9e4e9aad45a09
I20260812 17:48:30.049639 21119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:30.053625 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.048167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45522 (local address 127.13.38.62:40407)
0812 17:48:30.049527 (+ 1360us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:30.049545 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:30.049588 (+ 43us) server_negotiation.cc:407] Connection header received
0812 17:48:30.050424 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:30.050444 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:30.050835 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:30.051238 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:30.052185 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:30.052189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:30.052202 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:30.052212 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:30.052265 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:30.052282 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:30.052459 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:30.052554 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 17:48:30.052825 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":927,"thread_start_us":743,"threads_started":1}
I20260812 17:48:30.056286 21119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee: Bootstrap starting.
I20260812 17:48:30.058894 21119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:30.059886 21119 log.cc:824] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:30.062148 21119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee: No bootstrap required, opened a new log
I20260812 17:48:30.066416 21119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ebf5bae09c149598dab7c8401c7f8ee" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40407 } }
I20260812 17:48:30.066753 21119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:30.066855 21119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ebf5bae09c149598dab7c8401c7f8ee, State: Initialized, Role: FOLLOWER
I20260812 17:48:30.067368 21119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ebf5bae09c149598dab7c8401c7f8ee" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40407 } }
I20260812 17:48:30.067512 21119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:30.067634 21119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:30.067780 21119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:30.068814 21119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ebf5bae09c149598dab7c8401c7f8ee" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40407 } }
I20260812 17:48:30.069196 21119 leader_election.cc:302] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ebf5bae09c149598dab7c8401c7f8ee; no voters:
I20260812 17:48:30.069568 21119 leader_election.cc:288] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:30.069764 21125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:30.070045 21125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [term 1 LEADER]: Becoming Leader. State: Replica: 0ebf5bae09c149598dab7c8401c7f8ee, State: Running, Role: LEADER
I20260812 17:48:30.070374 21125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ebf5bae09c149598dab7c8401c7f8ee" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40407 } }
I20260812 17:48:30.070997 21119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:30.072225 21127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ebf5bae09c149598dab7c8401c7f8ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ebf5bae09c149598dab7c8401c7f8ee" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40407 } } }
I20260812 17:48:30.072381 21127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:30.072847 21129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ebf5bae09c149598dab7c8401c7f8ee. Latest consensus state: current_term: 1 leader_uuid: "0ebf5bae09c149598dab7c8401c7f8ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ebf5bae09c149598dab7c8401c7f8ee" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40407 } } }
I20260812 17:48:30.072960 21129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee [sys.catalog]: This master's current role is: LEADER
W20260812 17:48:30.076731 21155 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:48:30.076819 21155 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:48:30.076915 21145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:30.077636 21145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:30.083730 21145 catalog_manager.cc:1347] Generated new cluster ID: 4add40fe86cc44f7915b342a6a56b95b
I20260812 17:48:30.083834 21145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:30.099380 21145 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:30.099814 21145 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:30.107676 21145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee: Generated new TSK 0
I20260812 17:48:30.108563 21145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:48:30.113691 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:40407
--builtin_ntp_servers=127.13.38.20:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 17:48:30.250243 21162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:30.250537 21162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:30.250579 21162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:30.254568 21162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:30.254624 21162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:30.254680 21162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:30.259416 21162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:40407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:30.260567 21162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:30.261971 21162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:30.272282 21174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:30.272354 21171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:30.272403 21168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:30.272657 21162 server_base.cc:1034] running on GCE node
I20260812 17:48:30.273140 21162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:30.273841 21162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:30.275058 21162 hybrid_clock.cc:648] HybridClock initialized: now 1786556910275033 us; error 33 us; skew 500 ppm
I20260812 17:48:30.277204 21162 webserver.cc:460] Webserver started at http://127.13.38.1:35389/ using document root <none> and password file <none>
I20260812 17:48:30.277850 21162 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:30.277953 21162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:30.278204 21162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:30.280478 21162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "58e6b6574adc433087e7441daa2de194"
format_stamp: "Formatted at 2026-08-12 17:48:30 on dist-test-slave-sb2z"
server_key: "38133a6167966a12dfd70f050c495fa3"
server_key_iv: "4a3d520e5f0f12fe4e28a8b1ac79d3ea"
server_key_version: "encryptionkey@0"
I20260812 17:48:30.281111 21162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "58e6b6574adc433087e7441daa2de194"
format_stamp: "Formatted at 2026-08-12 17:48:30 on dist-test-slave-sb2z"
server_key: "38133a6167966a12dfd70f050c495fa3"
server_key_iv: "4a3d520e5f0f12fe4e28a8b1ac79d3ea"
server_key_version: "encryptionkey@0"
I20260812 17:48:30.285348 21162 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:48:30.288213 21182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:30.289716 21162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:30.289824 21162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "58e6b6574adc433087e7441daa2de194"
format_stamp: "Formatted at 2026-08-12 17:48:30 on dist-test-slave-sb2z"
server_key: "38133a6167966a12dfd70f050c495fa3"
server_key_iv: "4a3d520e5f0f12fe4e28a8b1ac79d3ea"
server_key_version: "encryptionkey@0"
I20260812 17:48:30.289927 21162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:30.294867 21162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:30.295502 21162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:30.295725 21162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:30.296269 21162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:30.297202 21162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:30.297250 21162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:30.297288 21162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:30.297304 21162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:30.307142 21162 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:35665
I20260812 17:48:30.307210 21357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:35665 every 8 connection(s)
I20260812 17:48:30.308238 21162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:30.313060 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 21162
I20260812 17:48:30.313212 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:30.313560 13464 external_mini_cluster.cc:1428] Setting key 1239104b4dbc4038f5fd252f26637589
I20260812 17:48:30.315280 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.310169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:45347 (local address 127.13.38.62:40407)
0812 17:48:30.310358 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:30.310364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:30.311506 (+ 1142us) server_negotiation.cc:407] Connection header received
0812 17:48:30.313196 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:30.313201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:30.313255 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:30.313374 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:30.314723 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:30.314727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:30.314730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:30.314742 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:30.314764 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:30.314774 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:30.314873 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:30.315088 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 17:48:30.315140 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 17:48:30.316037 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:40407
--builtin_ntp_servers=127.13.38.20:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 17:48:30.316475 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.310631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:40407 (local address 127.13.38.1:45347)
0812 17:48:30.311346 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:30.311388 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:30.312891 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:30.313626 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:30.313636 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:30.313756 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:30.314435 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:48:30.314436 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:30.314495 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:48:30.314568 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:30.314868 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:30.314875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:30.314879 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 17:48:30.315084 (+ 205us) client_negotiation.cc:240] Negotiation successful
0812 17:48:30.315338 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":122,"threads_started":1}
I20260812 17:48:30.318054 21358 heartbeater.cc:347] Connected to a master server at 127.13.38.62:40407
I20260812 17:48:30.318305 21358 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:30.318733 21358 heartbeater.cc:511] Master 127.13.38.62:40407 requested a full tablet report, sending...
I20260812 17:48:30.320386 21060 ts_manager.cc:194] Registered new tserver with Master: 58e6b6574adc433087e7441daa2de194 (127.13.38.1:35665)
W20260812 17:48:30.436542 21362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:30.436851 21362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:30.436932 21362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:30.440950 21362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:30.441042 21362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:30.441139 21362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:30.445917 21362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:40407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:30.447086 21362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:30.448611 21362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:30.458459 21375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:30.458459 21369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:30.458606 21367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:30.458880 21362 server_base.cc:1034] running on GCE node
I20260812 17:48:30.459272 21362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:30.460002 21362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:30.461184 21362 hybrid_clock.cc:648] HybridClock initialized: now 1786556910461169 us; error 29 us; skew 500 ppm
I20260812 17:48:30.463084 21362 webserver.cc:460] Webserver started at http://127.13.38.2:40545/ using document root <none> and password file <none>
I20260812 17:48:30.463780 21362 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:30.463851 21362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:30.464031 21362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:30.466329 21362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "aec5872e19844307b3a7491b8be6d52e"
format_stamp: "Formatted at 2026-08-12 17:48:30 on dist-test-slave-sb2z"
server_key: "0d91ce6947e5c9fc91b4defb64d42fc2"
server_key_iv: "8cc2027b033724f2f982859cd7ee832d"
server_key_version: "encryptionkey@0"
I20260812 17:48:30.466866 21362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "aec5872e19844307b3a7491b8be6d52e"
format_stamp: "Formatted at 2026-08-12 17:48:30 on dist-test-slave-sb2z"
server_key: "0d91ce6947e5c9fc91b4defb64d42fc2"
server_key_iv: "8cc2027b033724f2f982859cd7ee832d"
server_key_version: "encryptionkey@0"
I20260812 17:48:30.471105 21362 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:48:30.474116 21383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:30.475518 21362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:30.475705 21362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "aec5872e19844307b3a7491b8be6d52e"
format_stamp: "Formatted at 2026-08-12 17:48:30 on dist-test-slave-sb2z"
server_key: "0d91ce6947e5c9fc91b4defb64d42fc2"
server_key_iv: "8cc2027b033724f2f982859cd7ee832d"
server_key_version: "encryptionkey@0"
I20260812 17:48:30.475807 21362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:30.480471 21362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:30.481096 21362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:30.481243 21362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:30.481766 21362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:30.482690 21362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:30.482741 21362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:30.482781 21362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:30.482796 21362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:30.492061 21362 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:33429
I20260812 17:48:30.492121 21564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:33429 every 8 connection(s)
I20260812 17:48:30.493156 21362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:30.493954 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 21362
I20260812 17:48:30.494107 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:30.494467 13464 external_mini_cluster.cc:1428] Setting key 27bbe4436dcfe3d6bb9ef4d14efe05e8
I20260812 17:48:30.496968 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:40407
--builtin_ntp_servers=127.13.38.20:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 17:48:30.500656 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.495499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:55487 (local address 127.13.38.62:40407)
0812 17:48:30.495685 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:30.495690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:30.497164 (+ 1474us) server_negotiation.cc:407] Connection header received
0812 17:48:30.498592 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:30.498595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:30.498642 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:30.498734 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:30.500069 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:30.500073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:30.500079 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:30.500098 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:30.500126 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:30.500139 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:30.500249 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:30.500418 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 17:48:30.500471 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 17:48:30.501607 21568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.496147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:40407 (local address 127.13.38.2:55487)
0812 17:48:30.496999 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:30.497056 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:30.498357 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:30.498950 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:30.498958 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:30.499051 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:30.499734 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:48:30.499736 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:30.499787 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:48:30.499866 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:30.500253 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:30.500263 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:30.500268 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 17:48:30.500428 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 17:48:30.500649 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":160,"threads_started":1}
I20260812 17:48:30.502905 21566 heartbeater.cc:347] Connected to a master server at 127.13.38.62:40407
I20260812 17:48:30.503214 21566 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:30.503719 21566 heartbeater.cc:511] Master 127.13.38.62:40407 requested a full tablet report, sending...
I20260812 17:48:30.504882 21060 ts_manager.cc:194] Registered new tserver with Master: aec5872e19844307b3a7491b8be6d52e (127.13.38.2:33429)
W20260812 17:48:30.618461 21569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:30.618769 21569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:30.618813 21569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:30.623080 21569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:30.623163 21569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:30.623224 21569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:30.628357 21569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:40407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:30.629501 21569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:30.630918 21569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 17:48:30.641314 21569 server_base.cc:1034] running on GCE node
W20260812 17:48:30.641297 21580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:30.641271 21582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:30.641271 21585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:30.641990 21569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:30.642663 21569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:30.643854 21569 hybrid_clock.cc:648] HybridClock initialized: now 1786556910643816 us; error 50 us; skew 500 ppm
I20260812 17:48:30.646278 21569 webserver.cc:460] Webserver started at http://127.13.38.3:40431/ using document root <none> and password file <none>
I20260812 17:48:30.647202 21569 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:30.647321 21569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:30.647718 21569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:30.651002 21569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "45190373f58d4e47922d5ad6b7e36661"
format_stamp: "Formatted at 2026-08-12 17:48:30 on dist-test-slave-sb2z"
server_key: "f46dd1dc215f5cc1b6c072623d6a1e4c"
server_key_iv: "e9aff97132d4cfd8479aa06376408dd9"
server_key_version: "encryptionkey@0"
I20260812 17:48:30.651932 21569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "45190373f58d4e47922d5ad6b7e36661"
format_stamp: "Formatted at 2026-08-12 17:48:30 on dist-test-slave-sb2z"
server_key: "f46dd1dc215f5cc1b6c072623d6a1e4c"
server_key_iv: "e9aff97132d4cfd8479aa06376408dd9"
server_key_version: "encryptionkey@0"
I20260812 17:48:30.657613 21569 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 17:48:30.660496 21596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:30.661931 21569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:48:30.662089 21569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "45190373f58d4e47922d5ad6b7e36661"
format_stamp: "Formatted at 2026-08-12 17:48:30 on dist-test-slave-sb2z"
server_key: "f46dd1dc215f5cc1b6c072623d6a1e4c"
server_key_iv: "e9aff97132d4cfd8479aa06376408dd9"
server_key_version: "encryptionkey@0"
I20260812 17:48:30.662235 21569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:30.667346 21569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:30.668049 21569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:30.668247 21569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:30.668883 21569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:30.669876 21569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:30.669950 21569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:30.670027 21569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:30.670080 21569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:30.679790 21569 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:38257
I20260812 17:48:30.679865 21790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:38257 every 8 connection(s)
I20260812 17:48:30.680874 21569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:30.686085 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 21569
I20260812 17:48:30.686231 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:30.686594 13464 external_mini_cluster.cc:1428] Setting key de47fbf60b7576eb9cea584817403466
I20260812 17:48:30.687486 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.682814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:58949 (local address 127.13.38.62:40407)
0812 17:48:30.682971 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:30.682975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:30.684234 (+ 1259us) server_negotiation.cc:407] Connection header received
0812 17:48:30.685597 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:30.685600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:30.685647 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:30.685729 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:30.686957 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:30.686961 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:30.686964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:30.686979 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:30.687004 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:30.687020 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:30.687128 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:30.687282 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 17:48:30.687337 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 17:48:30.688519 21794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.683254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:40407 (local address 127.13.38.3:58949)
0812 17:48:30.684067 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:30.684111 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:30.685373 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:30.685898 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:30.685905 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:30.686013 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:30.686672 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:48:30.686675 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:30.686732 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:48:30.686806 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:30.687110 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:30.687120 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:30.687126 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 17:48:30.687316 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 17:48:30.687612 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":124,"threads_started":1}
I20260812 17:48:30.689805 21791 heartbeater.cc:347] Connected to a master server at 127.13.38.62:40407
I20260812 17:48:30.690058 21791 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:30.690514 21791 heartbeater.cc:511] Master 127.13.38.62:40407 requested a full tablet report, sending...
I20260812 17:48:30.691663 21060 ts_manager.cc:194] Registered new tserver with Master: 45190373f58d4e47922d5ad6b7e36661 (127.13.38.3:38257)
I20260812 17:48:30.702080 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:48:30.705786 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.704180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45536 (local address 127.13.38.62:40407)
0812 17:48:30.704315 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:30.704319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:30.704491 (+ 172us) server_negotiation.cc:407] Connection header received
0812 17:48:30.704603 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:30.704605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:30.704638 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:30.704709 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:30.705465 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:30.705469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:30.705472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:30.705481 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:30.705498 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:30.705505 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:30.705572 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:30.705645 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 17:48:30.705677 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 17:48:30.708681 21060 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:48:30.711817 21060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:30.724918 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.721205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:33429 (local address 127.0.0.1:42620)
0812 17:48:30.722203 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:30.722220 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:30.722309 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:30.723264 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:30.723273 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:30.723354 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:30.724006 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:48:30.724008 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:30.724030 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:48:30.724114 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:30.724512 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:30.724523 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:30.724533 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 17:48:30.724664 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 17:48:30.724759 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":922,"thread_start_us":121,"threads_started":1}
I20260812 17:48:30.725385 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.721908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:38257 (local address 127.0.0.1:36462)
0812 17:48:30.722353 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:30.722367 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:30.722447 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:30.724008 (+ 1561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:30.724012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:30.724027 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:30.724645 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:48:30.724647 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:30.724655 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:48:30.724682 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:30.725159 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:30.725162 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:30.725165 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 17:48:30.725237 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 17:48:30.725271 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":112,"threads_started":1}
I20260812 17:48:30.725560 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.722079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42620 (local address 127.13.38.2:33429)
0812 17:48:30.722747 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:30.722759 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:30.722784 (+ 25us) server_negotiation.cc:407] Connection header received
0812 17:48:30.722838 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:30.722842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:30.723080 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:30.723235 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:30.724284 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:30.724294 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:30.724309 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:30.724323 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:30.724376 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:30.724410 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:30.724537 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:30.725312 (+ 775us) server_negotiation.cc:299] Negotiation successful
0812 17:48:30.725382 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":149,"threads_started":1}
I20260812 17:48:30.726042 21814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.721098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:35665 (local address 127.0.0.1:41726)
0812 17:48:30.721664 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:30.721719 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:30.721837 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:30.724002 (+ 2165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:30.724005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:30.724021 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:30.724642 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:48:30.724644 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:30.724655 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:48:30.724682 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:30.725815 (+ 1133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:30.725819 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:30.725822 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 17:48:30.725899 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 17:48:30.725933 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":130,"threads_started":1}
I20260812 17:48:30.726161 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.722668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36462 (local address 127.13.38.3:38257)
0812 17:48:30.723045 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:30.723057 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:30.723079 (+ 22us) server_negotiation.cc:407] Connection header received
0812 17:48:30.723134 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:30.723141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:30.723298 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:30.723428 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:30.724859 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:30.724864 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:30.724873 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:30.724884 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:30.724949 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:30.724985 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:30.725116 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:30.725908 (+ 792us) server_negotiation.cc:299] Negotiation successful
0812 17:48:30.725982 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":155,"threads_started":1}
I20260812 17:48:30.726163 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.721661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41726 (local address 127.13.38.1:35665)
0812 17:48:30.722888 (+ 1227us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:30.722901 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:30.722926 (+ 25us) server_negotiation.cc:407] Connection header received
0812 17:48:30.722982 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:30.722988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:30.723154 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:30.723317 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:30.725584 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:30.725592 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:30.725604 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:30.725620 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:30.725672 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:30.725711 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:30.725833 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:30.725937 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 17:48:30.726015 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1104,"thread_start_us":123,"threads_started":1}
I20260812 17:48:30.728884 21259 tablet_service.cc:1467] Processing CreateTablet for tablet a0ba6e36d0a849c1b2c353f4929de803 (DEFAULT_TABLE table=test-table [id=8bb1228115f04156ba1e3bee1d7cb8e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:30.728884 21463 tablet_service.cc:1467] Processing CreateTablet for tablet a0ba6e36d0a849c1b2c353f4929de803 (DEFAULT_TABLE table=test-table [id=8bb1228115f04156ba1e3bee1d7cb8e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:30.728884 21683 tablet_service.cc:1467] Processing CreateTablet for tablet a0ba6e36d0a849c1b2c353f4929de803 (DEFAULT_TABLE table=test-table [id=8bb1228115f04156ba1e3bee1d7cb8e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:30.730017 21259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ba6e36d0a849c1b2c353f4929de803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:30.730021 21463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ba6e36d0a849c1b2c353f4929de803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:30.730021 21683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ba6e36d0a849c1b2c353f4929de803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:30.736965 21825 tablet_bootstrap.cc:492] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194: Bootstrap starting.
I20260812 17:48:30.737957 21828 tablet_bootstrap.cc:492] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661: Bootstrap starting.
I20260812 17:48:30.739763 21825 tablet_bootstrap.cc:654] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:30.739944 21827 tablet_bootstrap.cc:492] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e: Bootstrap starting.
I20260812 17:48:30.740134 21828 tablet_bootstrap.cc:654] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:30.740770 21825 log.cc:824] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:30.740911 21828 log.cc:824] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:30.742219 21827 tablet_bootstrap.cc:654] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:30.742954 21827 log.cc:824] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:30.743113 21825 tablet_bootstrap.cc:492] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194: No bootstrap required, opened a new log
I20260812 17:48:30.743113 21828 tablet_bootstrap.cc:492] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661: No bootstrap required, opened a new log
I20260812 17:48:30.743330 21828 ts_tablet_manager.cc:1397] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:48:30.743330 21825 ts_tablet_manager.cc:1397] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 17:48:30.745538 21827 tablet_bootstrap.cc:492] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e: No bootstrap required, opened a new log
I20260812 17:48:30.745772 21827 ts_tablet_manager.cc:1397] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:48:30.748371 21828 raft_consensus.cc:348] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e6b6574adc433087e7441daa2de194" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 35665 } } peers { permanent_uuid: "aec5872e19844307b3a7491b8be6d52e" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33429 } } peers { permanent_uuid: "45190373f58d4e47922d5ad6b7e36661" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 38257 } }
I20260812 17:48:30.748373 21825 raft_consensus.cc:348] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e6b6574adc433087e7441daa2de194" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 35665 } } peers { permanent_uuid: "aec5872e19844307b3a7491b8be6d52e" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33429 } } peers { permanent_uuid: "45190373f58d4e47922d5ad6b7e36661" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 38257 } }
I20260812 17:48:30.748697 21828 raft_consensus.cc:374] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:30.748697 21825 raft_consensus.cc:374] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:30.748764 21825 raft_consensus.cc:729] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58e6b6574adc433087e7441daa2de194, State: Initialized, Role: FOLLOWER
I20260812 17:48:30.748795 21828 raft_consensus.cc:729] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45190373f58d4e47922d5ad6b7e36661, State: Initialized, Role: FOLLOWER
I20260812 17:48:30.749327 21828 consensus_queue.cc:260] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e6b6574adc433087e7441daa2de194" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 35665 } } peers { permanent_uuid: "aec5872e19844307b3a7491b8be6d52e" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33429 } } peers { permanent_uuid: "45190373f58d4e47922d5ad6b7e36661" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 38257 } }
I20260812 17:48:30.749327 21825 consensus_queue.cc:260] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e6b6574adc433087e7441daa2de194" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 35665 } } peers { permanent_uuid: "aec5872e19844307b3a7491b8be6d52e" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33429 } } peers { permanent_uuid: "45190373f58d4e47922d5ad6b7e36661" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 38257 } }
I20260812 17:48:30.749531 21827 raft_consensus.cc:348] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e6b6574adc433087e7441daa2de194" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 35665 } } peers { permanent_uuid: "aec5872e19844307b3a7491b8be6d52e" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33429 } } peers { permanent_uuid: "45190373f58d4e47922d5ad6b7e36661" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 38257 } }
I20260812 17:48:30.749894 21827 raft_consensus.cc:374] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:30.750020 21827 raft_consensus.cc:729] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aec5872e19844307b3a7491b8be6d52e, State: Initialized, Role: FOLLOWER
I20260812 17:48:30.750366 21791 heartbeater.cc:503] Master 127.13.38.62:40407 was elected leader, sending a full tablet report...
I20260812 17:48:30.750491 21825 ts_tablet_manager.cc:1428] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 17:48:30.750491 21828 ts_tablet_manager.cc:1428] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 17:48:30.750614 21827 consensus_queue.cc:260] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e6b6574adc433087e7441daa2de194" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 35665 } } peers { permanent_uuid: "aec5872e19844307b3a7491b8be6d52e" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33429 } } peers { permanent_uuid: "45190373f58d4e47922d5ad6b7e36661" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 38257 } }
I20260812 17:48:30.750833 21358 heartbeater.cc:503] Master 127.13.38.62:40407 was elected leader, sending a full tablet report...
I20260812 17:48:30.751701 21827 ts_tablet_manager.cc:1428] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 17:48:30.752115 21566 heartbeater.cc:503] Master 127.13.38.62:40407 was elected leader, sending a full tablet report...
I20260812 17:48:30.775784 21837 raft_consensus.cc:482] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:30.776034 21837 raft_consensus.cc:504] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e6b6574adc433087e7441daa2de194" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 35665 } } peers { permanent_uuid: "aec5872e19844307b3a7491b8be6d52e" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33429 } } peers { permanent_uuid: "45190373f58d4e47922d5ad6b7e36661" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 38257 } }
I20260812 17:48:30.777217 21837 leader_election.cc:288] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aec5872e19844307b3a7491b8be6d52e (127.13.38.2:33429), 45190373f58d4e47922d5ad6b7e36661 (127.13.38.3:38257)
I20260812 17:48:30.779286 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.777662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:44009 (local address 127.13.38.2:33429)
0812 17:48:30.777844 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:30.777848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:30.777863 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:48:30.777904 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:30.777906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:30.777935 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:30.778007 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:30.778874 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:30.778877 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:30.778879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:30.778890 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:30.778909 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:30.778918 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:30.779024 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:30.779122 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 17:48:30.779161 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 17:48:30.779286 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.777422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:33429 (local address 127.13.38.1:44009)
0812 17:48:30.777558 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:30.777578 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:30.777671 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:30.778055 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:30.778058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:30.778074 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:30.778667 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:48:30.778672 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:30.778688 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:48:30.778724 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:30.779010 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:30.779013 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:30.779017 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 17:48:30.779112 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 17:48:30.779152 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 17:48:30.779323 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.777687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:51333 (local address 127.13.38.3:38257)
0812 17:48:30.777862 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:30.777867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:30.778063 (+ 196us) server_negotiation.cc:407] Connection header received
0812 17:48:30.778130 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:30.778132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:30.778162 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:30.778225 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:30.778895 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:30.778903 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:30.778906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:30.778919 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:30.778943 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:30.778994 (+ 51us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:30.779085 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:30.779152 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 17:48:30.779193 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 17:48:30.779346 21841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.777493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:38257 (local address 127.13.38.1:51333)
0812 17:48:30.777923 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:30.777934 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:30.778055 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:30.778275 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:30.778278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:30.778297 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:30.778736 (+ 439us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:48:30.778738 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:30.778746 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:48:30.778793 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:30.779072 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:30.779075 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:30.779078 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 17:48:30.779158 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 17:48:30.779193 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":137,"threads_started":1}
I20260812 17:48:30.780078 21494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba6e36d0a849c1b2c353f4929de803" candidate_uuid: "58e6b6574adc433087e7441daa2de194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aec5872e19844307b3a7491b8be6d52e" is_pre_election: true
I20260812 17:48:30.780092 21720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba6e36d0a849c1b2c353f4929de803" candidate_uuid: "58e6b6574adc433087e7441daa2de194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45190373f58d4e47922d5ad6b7e36661" is_pre_election: true
I20260812 17:48:30.780385 21720 raft_consensus.cc:2454] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58e6b6574adc433087e7441daa2de194 in term 0.
I20260812 17:48:30.780395 21494 raft_consensus.cc:2454] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58e6b6574adc433087e7441daa2de194 in term 0.
I20260812 17:48:30.780910 21183 leader_election.cc:302] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45190373f58d4e47922d5ad6b7e36661, 58e6b6574adc433087e7441daa2de194; no voters:
I20260812 17:48:30.781184 21837 raft_consensus.cc:2781] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:30.781286 21837 raft_consensus.cc:482] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:30.781333 21837 raft_consensus.cc:3037] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:30.782490 21837 raft_consensus.cc:504] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e6b6574adc433087e7441daa2de194" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 35665 } } peers { permanent_uuid: "aec5872e19844307b3a7491b8be6d52e" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33429 } } peers { permanent_uuid: "45190373f58d4e47922d5ad6b7e36661" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 38257 } }
I20260812 17:48:30.782853 21837 leader_election.cc:288] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [CANDIDATE]: Term 1 election: Requested vote from peers aec5872e19844307b3a7491b8be6d52e (127.13.38.2:33429), 45190373f58d4e47922d5ad6b7e36661 (127.13.38.3:38257)
I20260812 17:48:30.783264 21494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba6e36d0a849c1b2c353f4929de803" candidate_uuid: "58e6b6574adc433087e7441daa2de194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aec5872e19844307b3a7491b8be6d52e"
I20260812 17:48:30.783275 21720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba6e36d0a849c1b2c353f4929de803" candidate_uuid: "58e6b6574adc433087e7441daa2de194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45190373f58d4e47922d5ad6b7e36661"
I20260812 17:48:30.783418 21494 raft_consensus.cc:3037] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:30.783439 21720 raft_consensus.cc:3037] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:30.784921 21494 raft_consensus.cc:2454] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58e6b6574adc433087e7441daa2de194 in term 1.
I20260812 17:48:30.784915 21720 raft_consensus.cc:2454] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58e6b6574adc433087e7441daa2de194 in term 1.
I20260812 17:48:30.785250 21184 leader_election.cc:302] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58e6b6574adc433087e7441daa2de194, aec5872e19844307b3a7491b8be6d52e; no voters:
I20260812 17:48:30.785508 21837 raft_consensus.cc:2781] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:30.785778 21837 raft_consensus.cc:686] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 1 LEADER]: Becoming Leader. State: Replica: 58e6b6574adc433087e7441daa2de194, State: Running, Role: LEADER
I20260812 17:48:30.786172 21837 consensus_queue.cc:237] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e6b6574adc433087e7441daa2de194" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 35665 } } peers { permanent_uuid: "aec5872e19844307b3a7491b8be6d52e" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33429 } } peers { permanent_uuid: "45190373f58d4e47922d5ad6b7e36661" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 38257 } }
I20260812 17:48:30.789636 21056 catalog_manager.cc:5495] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58e6b6574adc433087e7441daa2de194 (127.13.38.1). New cstate: current_term: 1 leader_uuid: "58e6b6574adc433087e7441daa2de194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e6b6574adc433087e7441daa2de194" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 35665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aec5872e19844307b3a7491b8be6d52e" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45190373f58d4e47922d5ad6b7e36661" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 38257 } health_report { overall_health: UNKNOWN } } }
I20260812 17:48:30.808044 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.806322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41736 (local address 127.13.38.1:35665)
0812 17:48:30.806479 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:30.806483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:30.806499 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:48:30.806727 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:30.806730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:30.806801 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:30.806885 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:30.807662 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:30.807665 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:30.807667 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:30.807685 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:30.807704 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:30.807712 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:30.807801 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:30.807888 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 17:48:30.807929 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 17:48:30.809499 21360 tablet.cc:2369] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:48:30.815071 21494 raft_consensus.cc:1261] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e [term 1 FOLLOWER]: Refusing update from remote peer 58e6b6574adc433087e7441daa2de194: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:30.815058 21720 raft_consensus.cc:1261] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661 [term 1 FOLLOWER]: Refusing update from remote peer 58e6b6574adc433087e7441daa2de194: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:30.816023 21843 consensus_queue.cc:1035] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45190373f58d4e47922d5ad6b7e36661" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:30.816327 21837 consensus_queue.cc:1035] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aec5872e19844307b3a7491b8be6d52e" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:30.824466 21848 mvcc.cc:205] Tried to move back new op lower bound from 7317737106688987136 to 7317737106583396352. Current Snapshot: MvccSnapshot[applied={T|T < 7317737106688987136}]
I20260812 17:48:30.826179 21849 mvcc.cc:205] Tried to move back new op lower bound from 7317737106688987136 to 7317737106583396352. Current Snapshot: MvccSnapshot[applied={T|T < 7317737106688987136}]
I20260812 17:48:30.828199 21851 mvcc.cc:205] Tried to move back new op lower bound from 7317737106688987136 to 7317737106583396352. Current Snapshot: MvccSnapshot[applied={T|T < 7317737106688987136}]
I20260812 17:48:30.834435 21056 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:30.834651 21056 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:30.837529 21056 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ebf5bae09c149598dab7c8401c7f8ee: Sending DeleteTablet for 3 replicas of tablet a0ba6e36d0a849c1b2c353f4929de803
I20260812 17:48:30.838476 21463 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ba6e36d0a849c1b2c353f4929de803 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:30 UTC) from {username='slave'} at 127.0.0.1:42620
I20260812 17:48:30.838476 21683 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ba6e36d0a849c1b2c353f4929de803 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:30 UTC) from {username='slave'} at 127.0.0.1:36462
I20260812 17:48:30.838559 21259 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ba6e36d0a849c1b2c353f4929de803 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:30 UTC) from {username='slave'} at 127.0.0.1:41726
I20260812 17:48:30.839148 21863 tablet_replica.cc:330] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e: stopping tablet replica
I20260812 17:48:30.839309 21870 tablet_replica.cc:330] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194: stopping tablet replica
I20260812 17:48:30.839668 21870 raft_consensus.cc:2229] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:30.839664 21863 raft_consensus.cc:2229] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:30.839903 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 21162
I20260812 17:48:30.840119 21870 raft_consensus.cc:2258] T a0ba6e36d0a849c1b2c353f4929de803 P 58e6b6574adc433087e7441daa2de194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:30.840524 21867 tablet_replica.cc:330] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661: stopping tablet replica
I20260812 17:48:30.840797 21867 raft_consensus.cc:2229] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:30.841037 21863 raft_consensus.cc:2258] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:30.841145 21867 raft_consensus.cc:2258] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:30.844558 21863 ts_tablet_manager.cc:1905] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:30.844558 21867 ts_tablet_manager.cc:1905] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:30.847476 21863 ts_tablet_manager.cc:1918] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:30.847654 21863 log.cc:1196] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/a0ba6e36d0a849c1b2c353f4929de803
I20260812 17:48:30.847991 21863 ts_tablet_manager.cc:1939] T a0ba6e36d0a849c1b2c353f4929de803 P aec5872e19844307b3a7491b8be6d52e: Deleting consensus metadata
I20260812 17:48:30.848645 21867 ts_tablet_manager.cc:1918] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:30.848791 21867 log.cc:1196] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/a0ba6e36d0a849c1b2c353f4929de803
I20260812 17:48:30.849285 21867 ts_tablet_manager.cc:1939] T a0ba6e36d0a849c1b2c353f4929de803 P 45190373f58d4e47922d5ad6b7e36661: Deleting consensus metadata
I20260812 17:48:30.849431 21039 catalog_manager.cc:4841] TS aec5872e19844307b3a7491b8be6d52e (127.13.38.2:33429): tablet a0ba6e36d0a849c1b2c353f4929de803 (table test-table [id=8bb1228115f04156ba1e3bee1d7cb8e2]) successfully deleted
I20260812 17:48:30.850780 21038 catalog_manager.cc:4841] TS 45190373f58d4e47922d5ad6b7e36661 (127.13.38.3:38257): tablet a0ba6e36d0a849c1b2c353f4929de803 (table test-table [id=8bb1228115f04156ba1e3bee1d7cb8e2]) successfully deleted
I20260812 17:48:30.856638 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 21362
W20260812 17:48:30.856675 21038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:35665 (error 108)
I20260812 17:48:30.863616 21814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:30.863169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:35665 (local address 127.0.0.1:41748)
0812 17:48:30.863390 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:30.863474 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260812 17:48:30.863984 21038 catalog_manager.cc:4584] TS 58e6b6574adc433087e7441daa2de194 (127.13.38.1:35665): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a0ba6e36d0a849c1b2c353f4929de803: Network error: Client connection negotiation failed: client connection to 127.13.38.1:35665: connect: Connection refused (error 111)
I20260812 17:48:30.870322 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 21569
I20260812 17:48:30.884230 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 21010
2026-08-12T17:48:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1055 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 17:48:30.915162 13464 mini_oidc.cc:152] Starting JWKS server
I20260812 17:48:30.917157 13464 webserver.cc:460] Webserver started at https://127.0.0.1:33791/ using document root <none> and password file <none>
I20260812 17:48:30.917321 13464 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 17:48:30.917586 13464 webserver.cc:460] Webserver started at http://127.0.0.1:45245/ using document root <none> and password file <none>
2026-08-12T17:48:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:30Z Disabled control of system clock
I20260812 17:48:30.929126 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:42453
--webserver_interface=127.13.38.62
--webserver_port=0
--builtin_ntp_servers=127.13.38.20:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:42453
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33791/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/testchainca.cert with env {}
W20260812 17:48:31.050631 21878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:31.050916 21878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:31.050957 21878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:31.054940 21878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:31.055015 21878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:31.055061 21878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:31.055097 21878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:31.055124 21878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 17:48:31.055152 21878 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 17:48:31.055192 21878 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33791/jwks.json
I20260812 17:48:31.060685 21878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:42453
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:42453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33791/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:31.061795 21878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:31.062795 21878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:31.071823 21891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:31.071867 21889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:31.071998 21878 server_base.cc:1034] running on GCE node
W20260812 17:48:31.071856 21888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:31.072751 21878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:31.073925 21878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:31.075146 21878 hybrid_clock.cc:648] HybridClock initialized: now 1786556911075113 us; error 48 us; skew 500 ppm
I20260812 17:48:31.077353 21878 webserver.cc:460] Webserver started at http://127.13.38.62:41749/ using document root <none> and password file <none>
I20260812 17:48:31.077988 21878 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:31.078092 21878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:31.078339 21878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:31.080637 21878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "25af577dfed0423d8241aeee9ab20e22"
format_stamp: "Formatted at 2026-08-12 17:48:31 on dist-test-slave-sb2z"
server_key: "dfb2a2b85e2838c339dc9af197ba4535"
server_key_iv: "82e41905d321abb4d52994496335dd0c"
server_key_version: "encryptionkey@0"
I20260812 17:48:31.081350 21878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "25af577dfed0423d8241aeee9ab20e22"
format_stamp: "Formatted at 2026-08-12 17:48:31 on dist-test-slave-sb2z"
server_key: "dfb2a2b85e2838c339dc9af197ba4535"
server_key_iv: "82e41905d321abb4d52994496335dd0c"
server_key_version: "encryptionkey@0"
I20260812 17:48:31.085486 21878 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:48:31.088330 21902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:31.089663 21878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:48:31.089836 21878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "25af577dfed0423d8241aeee9ab20e22"
format_stamp: "Formatted at 2026-08-12 17:48:31 on dist-test-slave-sb2z"
server_key: "dfb2a2b85e2838c339dc9af197ba4535"
server_key_iv: "82e41905d321abb4d52994496335dd0c"
server_key_version: "encryptionkey@0"
I20260812 17:48:31.089967 21878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:31.121937 21878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:31.122874 21878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:31.123160 21878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:31.131045 21878 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:42453
I20260812 17:48:31.131081 21984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:42453 every 8 connection(s)
I20260812 17:48:31.132136 21878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:31.135641 21985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:31.136724 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 21878
I20260812 17:48:31.136854 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:31.137176 13464 external_mini_cluster.cc:1428] Setting key f5988892740212e913f6b0dbbd906f1f
I20260812 17:48:31.142253 21985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22: Bootstrap starting.
I20260812 17:48:31.144692 21985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:31.145453 21985 log.cc:824] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:31.146727 21989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:31.138762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50752 (local address 127.13.38.62:42453)
0812 17:48:31.139435 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:31.139451 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:31.139494 (+ 43us) server_negotiation.cc:407] Connection header received
0812 17:48:31.140228 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:31.140268 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:31.140677 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:31.141046 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:31.141490 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.142466 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.143350 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.143885 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.144065 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:31.144967 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:31.144972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:31.144985 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:31.145365 (+ 380us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:31.145420 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:31.145439 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:31.145526 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:31.145724 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 17:48:31.146007 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":148,"threads_started":1}
I20260812 17:48:31.147569 21985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22: No bootstrap required, opened a new log
I20260812 17:48:31.149602 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:48:31.151690 21985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25af577dfed0423d8241aeee9ab20e22" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42453 } }
I20260812 17:48:31.151949 21985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:31.152037 21985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25af577dfed0423d8241aeee9ab20e22, State: Initialized, Role: FOLLOWER
I20260812 17:48:31.152475 21985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25af577dfed0423d8241aeee9ab20e22" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42453 } }
I20260812 17:48:31.152647 21985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:31.152729 21985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:31.152849 21985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:31.153852 21985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25af577dfed0423d8241aeee9ab20e22" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42453 } }
I20260812 17:48:31.154222 21985 leader_election.cc:302] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25af577dfed0423d8241aeee9ab20e22; no voters:
I20260812 17:48:31.154601 21985 leader_election.cc:288] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:31.154677 21996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:31.154883 21996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [term 1 LEADER]: Becoming Leader. State: Replica: 25af577dfed0423d8241aeee9ab20e22, State: Running, Role: LEADER
I20260812 17:48:31.155184 21996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25af577dfed0423d8241aeee9ab20e22" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42453 } }
I20260812 17:48:31.155994 21985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:31.157017 21997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25af577dfed0423d8241aeee9ab20e22. Latest consensus state: current_term: 1 leader_uuid: "25af577dfed0423d8241aeee9ab20e22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25af577dfed0423d8241aeee9ab20e22" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42453 } } }
I20260812 17:48:31.157244 21997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:31.157063 21998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25af577dfed0423d8241aeee9ab20e22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25af577dfed0423d8241aeee9ab20e22" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42453 } } }
I20260812 17:48:31.157564 21998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:31.157687 22007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:31.160712 22007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:31.166356 22007 catalog_manager.cc:1347] Generated new cluster ID: b090f876c1bb4168939e2a09236772ff
I20260812 17:48:31.166440 22007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:31.182034 22007 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:31.183084 22007 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:31.194351 22007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25af577dfed0423d8241aeee9ab20e22: Generated new TSK 0
I20260812 17:48:31.195616 22007 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 17:48:31.242398 22028 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 17:48:31.242586 22028 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 17:48:31.243185 21989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:31.216136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50768 (local address 127.13.38.62:42453)
0812 17:48:31.216315 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:31.216321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:31.216551 (+ 230us) server_negotiation.cc:407] Connection header received
0812 17:48:31.216807 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:31.216810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:31.216871 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:31.217082 (+ 211us) server_negotiation.cc:226] Negotiated authn=JWT
0812 17:48:31.217417 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.218181 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.219490 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.219822 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.219923 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:31.220386 (+ 463us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 17:48:31.242622 (+ 22236us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 17:48:31.243021 (+ 399us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50768: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 17:48:31.243304 21989 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50768: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 17:48:31.244788 22027 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:31.215989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42453 (local address 127.0.0.1:50768)
0812 17:48:31.216437 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:31.216454 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:31.216612 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:31.217125 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:31.217128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:31.217138 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0812 17:48:31.217287 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:31.217294 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.218307 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.218311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:31.219330 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:31.219342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.219963 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.219967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:31.220062 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:31.220129 (+ 67us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 17:48:31.242949 (+ 22820us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 17:48:31.243316 (+ 367us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.38.62:42453: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":178,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.38.62:42453: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 17:48:31.245651 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 21878
2026-08-12T17:48:31Z chronyd exiting
I20260812 17:48:31.295377 13464 test_util.cc:175] -----------------------------------------------
I20260812 17:48:31.295482 13464 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786556880371321-13464-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (382 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 17:48:31.296778 13464 mini_oidc.cc:152] Starting JWKS server
I20260812 17:48:31.297643 13464 webserver.cc:460] Webserver started at https://127.0.0.1:42153/ using document root <none> and password file <none>
I20260812 17:48:31.297734 13464 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 17:48:31.297952 13464 webserver.cc:460] Webserver started at http://127.0.0.1:44599/ using document root <none> and password file <none>
2026-08-12T17:48:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:31Z Disabled control of system clock
I20260812 17:48:31.309372 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:39027
--webserver_interface=127.13.38.62
--webserver_port=0
--builtin_ntp_servers=127.13.38.20:39329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:39027
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42153/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/testchainca.cert with env {}
W20260812 17:48:31.429404 22037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:31.429726 22037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:31.429811 22037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:31.433812 22037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:31.433911 22037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:31.433940 22037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:31.433990 22037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:31.434029 22037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 17:48:31.434064 22037 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 17:48:31.434096 22037 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42153/jwks.json
I20260812 17:48:31.439695 22037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:39027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:39027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42153/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:31.440874 22037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:31.441953 22037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:31.450893 22045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:31.450902 22052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:31.450902 22042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:31.451331 22037 server_base.cc:1034] running on GCE node
I20260812 17:48:31.452008 22037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:31.453154 22037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:31.454380 22037 hybrid_clock.cc:648] HybridClock initialized: now 1786556911454360 us; error 31 us; skew 500 ppm
I20260812 17:48:31.456578 22037 webserver.cc:460] Webserver started at http://127.13.38.62:34075/ using document root <none> and password file <none>
I20260812 17:48:31.457194 22037 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:31.457262 22037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:31.457480 22037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:31.459684 22037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "4853b9a2bbed4a8e9691f825757e61ea"
format_stamp: "Formatted at 2026-08-12 17:48:31 on dist-test-slave-sb2z"
server_key: "3acfad252e93d9580c5f2e3590b08ce9"
server_key_iv: "55393a4531e9370a20913d3370baf030"
server_key_version: "encryptionkey@0"
I20260812 17:48:31.460282 22037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "4853b9a2bbed4a8e9691f825757e61ea"
format_stamp: "Formatted at 2026-08-12 17:48:31 on dist-test-slave-sb2z"
server_key: "3acfad252e93d9580c5f2e3590b08ce9"
server_key_iv: "55393a4531e9370a20913d3370baf030"
server_key_version: "encryptionkey@0"
I20260812 17:48:31.464500 22037 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 17:48:31.467250 22062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:31.468703 22037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:48:31.468842 22037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "4853b9a2bbed4a8e9691f825757e61ea"
format_stamp: "Formatted at 2026-08-12 17:48:31 on dist-test-slave-sb2z"
server_key: "3acfad252e93d9580c5f2e3590b08ce9"
server_key_iv: "55393a4531e9370a20913d3370baf030"
server_key_version: "encryptionkey@0"
I20260812 17:48:31.468961 22037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:31.488054 22037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:31.488834 22037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:31.489046 22037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:31.497227 22037 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:39027
I20260812 17:48:31.497239 22141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:39027 every 8 connection(s)
I20260812 17:48:31.498364 22037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:31.501976 22142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:31.506475 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 22037
I20260812 17:48:31.506582 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:31.506870 13464 external_mini_cluster.cc:1428] Setting key 10e5870f04b9f3722675041fba9aa6c3
I20260812 17:48:31.508231 22142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea: Bootstrap starting.
I20260812 17:48:31.510694 22142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:31.511713 22142 log.cc:824] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:31.514084 22142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea: No bootstrap required, opened a new log
I20260812 17:48:31.517194 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:31.508519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56134 (local address 127.13.38.62:39027)
0812 17:48:31.509198 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:31.509214 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:31.509250 (+ 36us) server_negotiation.cc:407] Connection header received
0812 17:48:31.509895 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:31.509911 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:31.510331 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:31.510634 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:31.511094 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.512242 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.513533 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.514045 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.514200 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:31.515103 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:31.515109 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:31.515123 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:31.515632 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:31.515697 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:31.515719 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:31.515805 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:31.516037 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 17:48:31.516397 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":167,"threads_started":1}
I20260812 17:48:31.518396 22142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4853b9a2bbed4a8e9691f825757e61ea" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39027 } }
I20260812 17:48:31.518640 22142 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:31.518724 22142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4853b9a2bbed4a8e9691f825757e61ea, State: Initialized, Role: FOLLOWER
I20260812 17:48:31.519237 22142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4853b9a2bbed4a8e9691f825757e61ea" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39027 } }
I20260812 17:48:31.519426 22142 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:31.519500 22142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:31.519661 22142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:31.520371 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:48:31.520956 22142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4853b9a2bbed4a8e9691f825757e61ea" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39027 } }
I20260812 17:48:31.521347 22142 leader_election.cc:302] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4853b9a2bbed4a8e9691f825757e61ea; no voters:
I20260812 17:48:31.521761 22142 leader_election.cc:288] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:31.521836 22148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:31.522065 22148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [term 1 LEADER]: Becoming Leader. State: Replica: 4853b9a2bbed4a8e9691f825757e61ea, State: Running, Role: LEADER
I20260812 17:48:31.522439 22148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4853b9a2bbed4a8e9691f825757e61ea" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39027 } }
I20260812 17:48:31.523206 22142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:31.524308 22150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4853b9a2bbed4a8e9691f825757e61ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4853b9a2bbed4a8e9691f825757e61ea" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39027 } } }
I20260812 17:48:31.524474 22149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4853b9a2bbed4a8e9691f825757e61ea. Latest consensus state: current_term: 1 leader_uuid: "4853b9a2bbed4a8e9691f825757e61ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4853b9a2bbed4a8e9691f825757e61ea" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39027 } } }
I20260812 17:48:31.524528 22150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:31.524551 22149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:31.525084 22160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:31.528347 22160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:31.534413 22160 catalog_manager.cc:1347] Generated new cluster ID: aeca58c19ac64ea2a1fc76a1e181f81d
I20260812 17:48:31.534502 22160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:31.551291 22160 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:31.552395 22160 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:31.558717 22160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4853b9a2bbed4a8e9691f825757e61ea: Generated new TSK 0
I20260812 17:48:31.559481 22160 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 17:48:31.591645 22182 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 17:48:31.591800 22182 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 17:48:31.592511 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:31.575932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56138 (local address 127.13.38.62:39027)
0812 17:48:31.576088 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:31.576094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:31.576320 (+ 226us) server_negotiation.cc:407] Connection header received
0812 17:48:31.576445 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:31.576448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:31.576500 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:31.576611 (+ 111us) server_negotiation.cc:226] Negotiated authn=JWT
0812 17:48:31.576916 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.577512 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.578763 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.579025 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.579110 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:31.579463 (+ 353us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 17:48:31.592125 (+ 12662us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42153/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 17:48:31.592374 (+ 249us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56138: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42153/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 17:48:31.592608 22146 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56138: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42153/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 17:48:31.593140 22181 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:31.575854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39027 (local address 127.0.0.1:56138)
0812 17:48:31.576178 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:31.576194 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:31.576322 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:31.576629 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:31.576632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:31.576640 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 17:48:31.576760 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:31.576766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.577644 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.577648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:31.578626 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:31.578637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.579151 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.579155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:31.579254 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:31.579285 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 17:48:31.592361 (+ 13076us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42153/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 17:48:31.592625 (+ 264us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.38.62:39027: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42153/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":123,"threads_started":1}
I20260812 17:48:31.593827 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 22037
2026-08-12T17:48:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (344 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 17:48:31.641489 13464 mini_oidc.cc:152] Starting JWKS server
I20260812 17:48:31.642410 13464 webserver.cc:460] Webserver started at https://127.0.0.1:33281/ using document root <none> and password file <none>
I20260812 17:48:31.642503 13464 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 17:48:31.642822 13464 webserver.cc:460] Webserver started at http://127.0.0.1:46529/ using document root <none> and password file <none>
2026-08-12T17:48:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:31Z Disabled control of system clock
I20260812 17:48:31.654146 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:43157
--webserver_interface=127.13.38.62
--webserver_port=0
--builtin_ntp_servers=127.13.38.20:36835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:43157
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33281/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 17:48:31.771166 22192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:31.771488 22192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:31.771612 22192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:31.775684 22192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:31.775767 22192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:31.775796 22192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:31.775837 22192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:31.775872 22192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 17:48:31.775913 22192 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 17:48:31.775943 22192 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33281/jwks.json
I20260812 17:48:31.780874 22192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:43157
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:43157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33281/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:31.782081 22192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:31.783573 22192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:31.792289 22199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:31.792367 22192 server_base.cc:1034] running on GCE node
W20260812 17:48:31.792289 22197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:31.792479 22201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:31.793216 22192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:31.794452 22192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:31.795624 22192 hybrid_clock.cc:648] HybridClock initialized: now 1786556911795603 us; error 34 us; skew 500 ppm
I20260812 17:48:31.797786 22192 webserver.cc:460] Webserver started at http://127.13.38.62:34407/ using document root <none> and password file <none>
I20260812 17:48:31.798424 22192 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:31.798527 22192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:31.798787 22192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:31.801182 22192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "249736985841444c8d54b1be367c7afe"
format_stamp: "Formatted at 2026-08-12 17:48:31 on dist-test-slave-sb2z"
server_key: "4edb39677e469c46b42dda4dd1ed2e95"
server_key_iv: "19558b56875df254633826e0e87f55fc"
server_key_version: "encryptionkey@0"
I20260812 17:48:31.801821 22192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "249736985841444c8d54b1be367c7afe"
format_stamp: "Formatted at 2026-08-12 17:48:31 on dist-test-slave-sb2z"
server_key: "4edb39677e469c46b42dda4dd1ed2e95"
server_key_iv: "19558b56875df254633826e0e87f55fc"
server_key_version: "encryptionkey@0"
I20260812 17:48:31.805938 22192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:48:31.808743 22215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:31.810125 22192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:31.810279 22192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "249736985841444c8d54b1be367c7afe"
format_stamp: "Formatted at 2026-08-12 17:48:31 on dist-test-slave-sb2z"
server_key: "4edb39677e469c46b42dda4dd1ed2e95"
server_key_iv: "19558b56875df254633826e0e87f55fc"
server_key_version: "encryptionkey@0"
I20260812 17:48:31.810406 22192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:31.825085 22192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:31.825883 22192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:31.826109 22192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:31.833809 22192 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:43157
I20260812 17:48:31.833837 22308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:43157 every 8 connection(s)
I20260812 17:48:31.834859 22192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:31.838631 22310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:31.841205 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 22192
I20260812 17:48:31.841357 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:31.841681 13464 external_mini_cluster.cc:1428] Setting key 64f1134d546cb66c9e07f067fbc704bf
I20260812 17:48:31.845992 22310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe: Bootstrap starting.
I20260812 17:48:31.848896 22310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:31.849745 22310 log.cc:824] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:31.851831 22313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:31.843273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59700 (local address 127.13.38.62:43157)
0812 17:48:31.844105 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:31.844123 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:31.844163 (+ 40us) server_negotiation.cc:407] Connection header received
0812 17:48:31.844981 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:31.845003 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:31.845455 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:31.845821 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:31.846329 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.847298 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.848307 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.848804 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.848993 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:31.849915 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:31.849922 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:31.849947 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:31.850358 (+ 411us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:31.850439 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:31.850462 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:31.850575 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:31.850710 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 17:48:31.850995 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":201,"threads_started":1}
I20260812 17:48:31.852258 22310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe: No bootstrap required, opened a new log
I20260812 17:48:31.854957 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:48:31.856601 22310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "249736985841444c8d54b1be367c7afe" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43157 } }
I20260812 17:48:31.856897 22310 raft_consensus.cc:374] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:31.856981 22310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 249736985841444c8d54b1be367c7afe, State: Initialized, Role: FOLLOWER
I20260812 17:48:31.857461 22310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "249736985841444c8d54b1be367c7afe" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43157 } }
I20260812 17:48:31.857617 22310 raft_consensus.cc:388] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:31.857690 22310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:31.857792 22310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:31.858979 22310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "249736985841444c8d54b1be367c7afe" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43157 } }
I20260812 17:48:31.859352 22310 leader_election.cc:302] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 249736985841444c8d54b1be367c7afe; no voters:
I20260812 17:48:31.859717 22310 leader_election.cc:288] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:31.859882 22319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:31.860147 22319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [term 1 LEADER]: Becoming Leader. State: Replica: 249736985841444c8d54b1be367c7afe, State: Running, Role: LEADER
I20260812 17:48:31.860526 22319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "249736985841444c8d54b1be367c7afe" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43157 } }
I20260812 17:48:31.861063 22310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:31.862262 22325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 249736985841444c8d54b1be367c7afe. Latest consensus state: current_term: 1 leader_uuid: "249736985841444c8d54b1be367c7afe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "249736985841444c8d54b1be367c7afe" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43157 } } }
I20260812 17:48:31.862404 22325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:31.862692 22323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "249736985841444c8d54b1be367c7afe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "249736985841444c8d54b1be367c7afe" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 43157 } } }
I20260812 17:48:31.862792 22323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:31.863440 22334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:31.866168 22334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:31.871781 22334 catalog_manager.cc:1347] Generated new cluster ID: 81ff2b7c57a548c6ba71cb622ba754fb
I20260812 17:48:31.871867 22334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:31.880264 22334 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:31.881567 22334 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:31.892372 22334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 249736985841444c8d54b1be367c7afe: Generated new TSK 0
I20260812 17:48:31.893306 22334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:48:31.911118 22313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:31.905419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.13.38.62:43157)
0812 17:48:31.905578 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:31.905583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:31.905859 (+ 276us) server_negotiation.cc:407] Connection header received
0812 17:48:31.906089 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:31.906092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:31.906149 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:31.906258 (+ 109us) server_negotiation.cc:226] Negotiated authn=JWT
0812 17:48:31.906585 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.907267 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.908611 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.908837 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.908936 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:31.909298 (+ 362us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 17:48:31.910684 (+ 1386us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33281/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 17:48:31.910987 (+ 303us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59708: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33281/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 17:48:31.911218 22313 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59708: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33281/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 17:48:31.914162 22362 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:31.905260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:43157 (local address 127.0.0.1:59708)
0812 17:48:31.905744 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:31.905760 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:31.905891 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:31.906285 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:31.906290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:31.906302 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0812 17:48:31.906433 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:31.906443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.907400 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.907404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:31.908455 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:31.908467 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:31.908981 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:31.908985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:31.909081 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:31.909112 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 17:48:31.910873 (+ 1761us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33281/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 17:48:31.914001 (+ 3128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.38.62:43157: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33281/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":2849,"thread_start_us":120,"threads_started":1}
I20260812 17:48:31.914960 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 22192
E20260812 17:48:31.932928 22364 webserver.cc:533] Webserver: sslize failed:
E20260812 17:48:31.933058 22364 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T17:48:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (326 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T17:48:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:31Z Disabled control of system clock
I20260812 17:48:31.979027 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:37283
--webserver_interface=127.13.38.62
--webserver_port=0
--builtin_ntp_servers=127.13.38.20:41519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:37283
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/testchainca.cert with env {}
W20260812 17:48:32.100273 22369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:32.100569 22369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:32.100611 22369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:32.104636 22369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:32.104707 22369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:32.104723 22369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:32.104740 22369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:32.104755 22369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 17:48:32.104771 22369 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 17:48:32.104784 22369 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 17:48:32.110460 22369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:41519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:37283
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:37283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:32.111593 22369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:32.112614 22369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:32.121788 22381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:32.121788 22379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:32.121788 22384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:32.122169 22369 server_base.cc:1034] running on GCE node
I20260812 17:48:32.123035 22369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:32.124372 22369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:32.125602 22369 hybrid_clock.cc:648] HybridClock initialized: now 1786556912125584 us; error 41 us; skew 500 ppm
I20260812 17:48:32.128333 22369 webserver.cc:460] Webserver started at http://127.13.38.62:36917/ using document root <none> and password file <none>
I20260812 17:48:32.129181 22369 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:32.129294 22369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:32.129601 22369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:32.132179 22369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "524f0510237c4680b3c3c924f87da9f1"
format_stamp: "Formatted at 2026-08-12 17:48:32 on dist-test-slave-sb2z"
server_key: "e8d27f031ba2e9de7075984fa519c623"
server_key_iv: "9f166ba535eab6ce13767fe269771ec0"
server_key_version: "encryptionkey@0"
I20260812 17:48:32.132817 22369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "524f0510237c4680b3c3c924f87da9f1"
format_stamp: "Formatted at 2026-08-12 17:48:32 on dist-test-slave-sb2z"
server_key: "e8d27f031ba2e9de7075984fa519c623"
server_key_iv: "9f166ba535eab6ce13767fe269771ec0"
server_key_version: "encryptionkey@0"
I20260812 17:48:32.137068 22369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:48:32.139811 22394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:32.141176 22369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:32.141338 22369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "524f0510237c4680b3c3c924f87da9f1"
format_stamp: "Formatted at 2026-08-12 17:48:32 on dist-test-slave-sb2z"
server_key: "e8d27f031ba2e9de7075984fa519c623"
server_key_iv: "9f166ba535eab6ce13767fe269771ec0"
server_key_version: "encryptionkey@0"
I20260812 17:48:32.141463 22369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:32.166246 22369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:32.167189 22369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:32.167423 22369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:32.176131 22369 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:37283
I20260812 17:48:32.176147 22475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:37283 every 8 connection(s)
I20260812 17:48:32.177201 22369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:32.180368 22476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:32.186352 22476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1: Bootstrap starting.
I20260812 17:48:32.187124 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 22369
I20260812 17:48:32.187238 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:32.187642 13464 external_mini_cluster.cc:1428] Setting key c2f855293188c3f45a5fb2658f33ec09
I20260812 17:48:32.189509 22476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:32.190312 22476 log.cc:824] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:32.192724 22476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1: No bootstrap required, opened a new log
I20260812 17:48:32.197343 22476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "524f0510237c4680b3c3c924f87da9f1" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37283 } }
I20260812 17:48:32.197610 22476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:32.197697 22476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524f0510237c4680b3c3c924f87da9f1, State: Initialized, Role: FOLLOWER
I20260812 17:48:32.198084 22483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:32.189048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.13.38.62:37283)
0812 17:48:32.189832 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:32.189848 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:32.189888 (+ 40us) server_negotiation.cc:407] Connection header received
0812 17:48:32.190669 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:32.190691 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:32.191211 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:32.191634 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:32.192023 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:32.193135 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:32.194366 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:32.194868 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:32.195003 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:32.196026 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:32.196031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:32.196046 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:48:32.196462 (+ 416us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:32.196523 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:48:32.196544 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:32.196636 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:32.196810 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 17:48:32.197165 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":162,"threads_started":1}
I20260812 17:48:32.198263 22476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "524f0510237c4680b3c3c924f87da9f1" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37283 } }
I20260812 17:48:32.198452 22476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:32.198529 22476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:32.198644 22476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:32.199863 22476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "524f0510237c4680b3c3c924f87da9f1" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37283 } }
I20260812 17:48:32.200273 22476 leader_election.cc:302] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 524f0510237c4680b3c3c924f87da9f1; no voters:
I20260812 17:48:32.200707 22476 leader_election.cc:288] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:32.200834 22489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:32.201179 22489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [term 1 LEADER]: Becoming Leader. State: Replica: 524f0510237c4680b3c3c924f87da9f1, State: Running, Role: LEADER
I20260812 17:48:32.201396 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:48:32.201512 22489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "524f0510237c4680b3c3c924f87da9f1" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37283 } }
I20260812 17:48:32.202129 22476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:32.202687 22489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 524f0510237c4680b3c3c924f87da9f1. Latest consensus state: current_term: 1 leader_uuid: "524f0510237c4680b3c3c924f87da9f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "524f0510237c4680b3c3c924f87da9f1" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37283 } } }
I20260812 17:48:32.202831 22489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:32.203289 22492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "524f0510237c4680b3c3c924f87da9f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "524f0510237c4680b3c3c924f87da9f1" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37283 } } }
I20260812 17:48:32.203393 22492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:32.203850 22498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:32.207048 22498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:32.212903 22498 catalog_manager.cc:1347] Generated new cluster ID: e63a178203e04430951287d264063103
I20260812 17:48:32.213047 22498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:32.234067 22498 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:32.235142 22498 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:32.240656 22498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 524f0510237c4680b3c3c924f87da9f1: Generated new TSK 0
I20260812 17:48:32.241427 22498 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:48:32.284981 22483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:32.257240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.13.38.62:37283)
0812 17:48:32.257404 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:32.257411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:32.257575 (+ 164us) server_negotiation.cc:407] Connection header received
0812 17:48:32.257757 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:32.257760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:32.257810 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:32.257908 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0812 17:48:32.258188 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:32.258746 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:32.260046 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:32.260297 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:32.260407 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:32.260757 (+ 350us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 17:48:32.284545 (+ 23788us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 17:48:32.284842 (+ 297us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43204: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 17:48:32.285104 22483 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43204: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 17:48:32.285578 22526 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:32.257141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:37283 (local address 127.0.0.1:43204)
0812 17:48:32.257473 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:32.257490 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:32.257620 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:32.257933 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:32.257936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:32.257944 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 17:48:32.258071 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:32.258078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:32.258864 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:32.258867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:32.259861 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:32.259873 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:32.260436 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:32.260440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:32.260528 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:32.260566 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 17:48:32.284819 (+ 24253us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 17:48:32.285046 (+ 227us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.38.62:37283: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":119,"threads_started":1}
I20260812 17:48:32.286242 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 22369
2026-08-12T17:48:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (344 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (134 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (134 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:32 dist-test-slave-sb2z krb5kdc[22545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:32 dist-test-slave-sb2z krb5kdc[22545](info): set up 2 sockets
Aug 12 17:48:32 dist-test-slave-sb2z krb5kdc[22545](info): commencing operation
krb5kdc: starting...
W20260812 17:48:33.823027 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:33 dist-test-slave-sb2z krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:48:33.967124 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:37903
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:37903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:34.088603 22561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:34.088973 22561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:34.089063 22561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:34.093255 22561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:34.093377 22561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:34.093416 22561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:34.093456 22561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:34.093492 22561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:34.099098 22561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:37903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:37903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:34.100361 22561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:34.101536 22561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:34.111083 22568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:34.111056 22567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:34.111223 22561 server_base.cc:1034] running on GCE node
W20260812 17:48:34.111449 22574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:34.112085 22561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:34.113286 22561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:34.114502 22561 hybrid_clock.cc:648] HybridClock initialized: now 1786556914114478 us; error 36 us; skew 500 ppm
Aug 12 17:48:34 dist-test-slave-sb2z krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:34.117620 22561 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:48:34.118673 22561 webserver.cc:460] Webserver started at http://127.13.38.62:46275/ using document root <none> and password file <none>
I20260812 17:48:34.119294 22561 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:34.119401 22561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:34.119814 22561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:34.122084 22561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "50b0285e5d9243bab783bcf989eff023"
format_stamp: "Formatted at 2026-08-12 17:48:34 on dist-test-slave-sb2z"
server_key: "3ac5e3740992a58511b6a1e0e559df8e"
server_key_iv: "7856b2b77786bccbc31afcfafd30d42b"
server_key_version: "encryptionkey@0"
I20260812 17:48:34.122746 22561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "50b0285e5d9243bab783bcf989eff023"
format_stamp: "Formatted at 2026-08-12 17:48:34 on dist-test-slave-sb2z"
server_key: "3ac5e3740992a58511b6a1e0e559df8e"
server_key_iv: "7856b2b77786bccbc31afcfafd30d42b"
server_key_version: "encryptionkey@0"
I20260812 17:48:34.127252 22561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:48:34.130019 22585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:34.131430 22561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:34.131639 22561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "50b0285e5d9243bab783bcf989eff023"
format_stamp: "Formatted at 2026-08-12 17:48:34 on dist-test-slave-sb2z"
server_key: "3ac5e3740992a58511b6a1e0e559df8e"
server_key_iv: "7856b2b77786bccbc31afcfafd30d42b"
server_key_version: "encryptionkey@0"
I20260812 17:48:34.131778 22561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:34.147279 22561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:34.150393 22561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:34.150619 22561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:34.158169 22561 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:37903
I20260812 17:48:34.158190 22668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:37903 every 8 connection(s)
I20260812 17:48:34.159197 22561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:34.162148 22669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:34.165210 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 22561
I20260812 17:48:34.165361 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:34.165673 13464 external_mini_cluster.cc:1428] Setting key 10efc95e23b88faf3b9c8bcacf73f5a4
I20260812 17:48:34.168919 22669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023: Bootstrap starting.
I20260812 17:48:34.171293 22669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:34.172204 22669 log.cc:824] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:34.174775 22669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023: No bootstrap required, opened a new log
Aug 12 17:48:34 dist-test-slave-sb2z krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:34.179833 22669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b0285e5d9243bab783bcf989eff023" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37903 } }
I20260812 17:48:34.180181 22669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:34.180290 22669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50b0285e5d9243bab783bcf989eff023, State: Initialized, Role: FOLLOWER
I20260812 17:48:34.180902 22669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b0285e5d9243bab783bcf989eff023" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37903 } }
I20260812 17:48:34.181083 22669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:34.181150 22669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:34.181262 22669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:34.182595 22669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b0285e5d9243bab783bcf989eff023" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37903 } }
I20260812 17:48:34.182978 22669 leader_election.cc:302] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50b0285e5d9243bab783bcf989eff023; no voters:
I20260812 17:48:34.183230 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:34.167261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43922 (local address 127.13.38.62:37903)
0812 17:48:34.167769 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:34.167786 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:34.167824 (+ 38us) server_negotiation.cc:407] Connection header received
0812 17:48:34.168491 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:34.168510 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:34.168859 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:34.169251 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:34.170101 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.171120 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.172148 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.172751 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.172928 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:34.176108 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:34.176115 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:34.176132 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:34.176168 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:34.179388 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:34.180010 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:34.180017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:34.180024 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:34.180180 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:34.180569 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:34.180573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:34.180575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:34.181172 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:34.181401 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:34.181780 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 17:48:34.182120 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":157,"threads_started":1}
I20260812 17:48:34.183346 22669 leader_election.cc:288] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:34.183595 22676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:34.183856 22676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [term 1 LEADER]: Becoming Leader. State: Replica: 50b0285e5d9243bab783bcf989eff023, State: Running, Role: LEADER
I20260812 17:48:34.184229 22676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b0285e5d9243bab783bcf989eff023" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37903 } }
I20260812 17:48:34.184789 22669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:34.185564 22676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50b0285e5d9243bab783bcf989eff023. Latest consensus state: current_term: 1 leader_uuid: "50b0285e5d9243bab783bcf989eff023" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b0285e5d9243bab783bcf989eff023" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37903 } } }
I20260812 17:48:34.185694 22676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:34.186035 22679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50b0285e5d9243bab783bcf989eff023" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b0285e5d9243bab783bcf989eff023" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37903 } } }
I20260812 17:48:34.186105 22679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023 [sys.catalog]: This master's current role is: LEADER
W20260812 17:48:34.189857 22699 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:48:34.189985 22699 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:48:34.190064 22689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:34.190778 22689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:34.197088 22689 catalog_manager.cc:1347] Generated new cluster ID: 6659d04da6e74d778ce497b6c704796a
I20260812 17:48:34.197175 22689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:34.211571 22689 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:34.212916 22689 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:34.223088 22689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50b0285e5d9243bab783bcf989eff023: Generated new TSK 0
I20260812 17:48:34.224061 22689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:48:34.259323 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:37903
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:34.382956 22706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:34.383284 22706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:34.383363 22706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:34.387950 22706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:34.388080 22706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:34.388190 22706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:34.393733 22706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:37903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:34.394951 22706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:34.396090 22706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:34.406190 22719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:34.406198 22716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:34.406167 22714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:34.406508 22706 server_base.cc:1034] running on GCE node
I20260812 17:48:34.406948 22706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:34.407694 22706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:34.408888 22706 hybrid_clock.cc:648] HybridClock initialized: now 1786556914408871 us; error 35 us; skew 500 ppm
Aug 12 17:48:34 dist-test-slave-sb2z krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:34.411854 22706 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:48:34.412768 22706 webserver.cc:460] Webserver started at http://127.13.38.1:33809/ using document root <none> and password file <none>
I20260812 17:48:34.413360 22706 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:34.413460 22706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:34.413719 22706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:34.415906 22706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "3b6e98c990cc46ae8087b2cdfee65e1f"
format_stamp: "Formatted at 2026-08-12 17:48:34 on dist-test-slave-sb2z"
server_key: "576f32aa61b3edd03137d3b092961246"
server_key_iv: "be5ed67122a0d2c98c56d690cf35a70a"
server_key_version: "encryptionkey@0"
I20260812 17:48:34.416497 22706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "3b6e98c990cc46ae8087b2cdfee65e1f"
format_stamp: "Formatted at 2026-08-12 17:48:34 on dist-test-slave-sb2z"
server_key: "576f32aa61b3edd03137d3b092961246"
server_key_iv: "be5ed67122a0d2c98c56d690cf35a70a"
server_key_version: "encryptionkey@0"
I20260812 17:48:34.420617 22706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:48:34.423264 22733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:34.424676 22706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:34.424830 22706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "3b6e98c990cc46ae8087b2cdfee65e1f"
format_stamp: "Formatted at 2026-08-12 17:48:34 on dist-test-slave-sb2z"
server_key: "576f32aa61b3edd03137d3b092961246"
server_key_iv: "be5ed67122a0d2c98c56d690cf35a70a"
server_key_version: "encryptionkey@0"
I20260812 17:48:34.424957 22706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:34.445176 22706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:34.449012 22706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:34.449252 22706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:34.449944 22706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:34.450876 22706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:34.450954 22706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:34.451035 22706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:34.451078 22706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:34.460973 22706 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:46531
I20260812 17:48:34.461050 22906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:46531 every 8 connection(s)
I20260812 17:48:34.461992 22706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:34.468228 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 22706
I20260812 17:48:34.468379 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:34.468683 13464 external_mini_cluster.cc:1428] Setting key 7d4518804b99c7fa1b1df99ab8bc386c
Aug 12 17:48:34 dist-test-slave-sb2z krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:34.478713 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:34.464249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:49643 (local address 127.13.38.62:37903)
0812 17:48:34.464443 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:34.464448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:34.465302 (+ 854us) server_negotiation.cc:407] Connection header received
0812 17:48:34.466817 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:34.466820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:34.466885 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:34.466997 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:34.468541 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.469212 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.470836 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.471091 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.471184 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:34.474178 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:34.474184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:34.474190 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:34.474227 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:34.476150 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:34.476740 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:34.476745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:34.476749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:34.476856 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:34.477374 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:34.477377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:34.477378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:34.477572 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:34.477654 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:34.478468 (+ 814us) server_negotiation.cc:299] Negotiation successful
0812 17:48:34.478584 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 17:48:34.480228 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:34.464587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:37903 (local address 127.13.38.1:49643)
0812 17:48:34.465125 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:34.465173 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:34.466567 (+ 1394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:34.467196 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:34.467203 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:34.467627 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:34.468292 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:34.468317 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.469370 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.469374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:34.470679 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:34.470699 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.471268 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.471275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:34.471547 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:34.472280 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:34.472307 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:34.473989 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:34.476303 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:34.476312 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:34.476332 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:34.476625 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:34.477001 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:34.477009 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:34.477014 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:34.477253 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:34.477703 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:34.477713 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:34.478154 (+ 441us) client_negotiation.cc:769] Sending connection context
0812 17:48:34.478469 (+ 315us) client_negotiation.cc:240] Negotiation successful
0812 17:48:34.478920 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:48:34.481834 22908 heartbeater.cc:347] Connected to a master server at 127.13.38.62:37903
I20260812 17:48:34.482137 22908 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:34.482752 22908 heartbeater.cc:511] Master 127.13.38.62:37903 requested a full tablet report, sending...
I20260812 17:48:34.484608 22611 ts_manager.cc:194] Registered new tserver with Master: 3b6e98c990cc46ae8087b2cdfee65e1f (127.13.38.1:46531)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
I20260812 17:48:34.485985 22611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:49643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:48:34.504211 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:37903
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:34.625142 22916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:34.625481 22916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:34.625568 22916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:34.630023 22916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:34.630126 22916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:34.630218 22916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:34.635922 22916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:37903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:34.637190 22916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:34.638307 22916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:34.648916 22930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:34.649176 22916 server_base.cc:1034] running on GCE node
W20260812 17:48:34.649008 22926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:34.648931 22928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:34.649755 22916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:34.650456 22916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:34.651614 22916 hybrid_clock.cc:648] HybridClock initialized: now 1786556914651596 us; error 35 us; skew 500 ppm
Aug 12 17:48:34 dist-test-slave-sb2z krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:34.663235 22916 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:48:34.664642 22916 webserver.cc:460] Webserver started at http://127.13.38.2:34235/ using document root <none> and password file <none>
I20260812 17:48:34.665313 22916 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:34.665417 22916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:34.665684 22916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:34.668020 22916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "e17312b7692e49ff8b9b25bab1d47022"
format_stamp: "Formatted at 2026-08-12 17:48:34 on dist-test-slave-sb2z"
server_key: "f57ec936e77de128ff2585ff82cecd1c"
server_key_iv: "2442926d48cc78decfabed2b7078ee5a"
server_key_version: "encryptionkey@0"
I20260812 17:48:34.668722 22916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "e17312b7692e49ff8b9b25bab1d47022"
format_stamp: "Formatted at 2026-08-12 17:48:34 on dist-test-slave-sb2z"
server_key: "f57ec936e77de128ff2585ff82cecd1c"
server_key_iv: "2442926d48cc78decfabed2b7078ee5a"
server_key_version: "encryptionkey@0"
I20260812 17:48:34.673004 22916 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:48:34.675926 22939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:34.677317 22916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:48:34.677488 22916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "e17312b7692e49ff8b9b25bab1d47022"
format_stamp: "Formatted at 2026-08-12 17:48:34 on dist-test-slave-sb2z"
server_key: "f57ec936e77de128ff2585ff82cecd1c"
server_key_iv: "2442926d48cc78decfabed2b7078ee5a"
server_key_version: "encryptionkey@0"
I20260812 17:48:34.677629 22916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:34.690644 22916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:34.694069 22916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:34.694288 22916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:34.694918 22916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:34.695866 22916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:34.695938 22916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:34.696027 22916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:34.696070 22916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:34.705556 22916 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:35243
I20260812 17:48:34.705585 23116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:35243 every 8 connection(s)
I20260812 17:48:34.706470 22916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:34.712770 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 22916
I20260812 17:48:34.712922 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:34.713227 13464 external_mini_cluster.cc:1428] Setting key df54e31ccd57cb02d50fafd5a8e4e736
Aug 12 17:48:34 dist-test-slave-sb2z krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:34.721762 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:34.708593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:54441 (local address 127.13.38.62:37903)
0812 17:48:34.708773 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:34.708778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:34.709632 (+ 854us) server_negotiation.cc:407] Connection header received
0812 17:48:34.710993 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:34.710996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:34.711045 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:34.711124 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:34.712482 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.713321 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.714742 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.714975 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.715094 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:34.717742 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:34.717745 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:34.717747 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:34.717770 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:34.719231 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:34.719987 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:34.719990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:34.719991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:34.720068 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:34.720512 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:34.720514 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:34.720515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:34.720696 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:34.720770 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:34.721528 (+ 758us) server_negotiation.cc:299] Negotiation successful
0812 17:48:34.721637 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 17:48:34.723155 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:34.708959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:37903 (local address 127.13.38.2:54441)
0812 17:48:34.709462 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:34.709510 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:34.710790 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:34.711301 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:34.711308 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:34.711783 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:34.712343 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:34.712358 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.713490 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.713493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:34.714618 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:34.714628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.715143 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.715147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:34.715323 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:34.716050 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:34.716067 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:34.717558 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:34.719360 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:34.719370 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:34.719422 (+ 52us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:34.719882 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:34.720190 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:34.720194 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:34.720198 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:34.720400 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:34.720816 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:34.720826 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:34.721232 (+ 406us) client_negotiation.cc:769] Sending connection context
0812 17:48:34.721501 (+ 269us) client_negotiation.cc:240] Negotiation successful
0812 17:48:34.721928 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":121,"threads_started":1}
I20260812 17:48:34.724586 23118 heartbeater.cc:347] Connected to a master server at 127.13.38.62:37903
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:48:34.724865 23118 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:34.725441 23118 heartbeater.cc:511] Master 127.13.38.62:37903 requested a full tablet report, sending...
I20260812 17:48:34.726648 22611 ts_manager.cc:194] Registered new tserver with Master: e17312b7692e49ff8b9b25bab1d47022 (127.13.38.2:35243)
I20260812 17:48:34.727306 22611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:54441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:48:34.747413 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:37903
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:34.881326 23123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:34.881719 23123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:34.881807 23123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:34.888908 23123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:34.889007 23123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:34.889113 23123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:34.895478 23123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:37903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:34.896984 23123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:34.898049 23123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:34.908280 23133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:34.908339 23123 server_base.cc:1034] running on GCE node
W20260812 17:48:34.908280 23137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:34.908566 23134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:34.909037 23123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:34.909744 23123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:34.910938 23123 hybrid_clock.cc:648] HybridClock initialized: now 1786556914910894 us; error 48 us; skew 500 ppm
Aug 12 17:48:34 dist-test-slave-sb2z krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:34.914170 23123 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:48:34.915249 23123 webserver.cc:460] Webserver started at http://127.13.38.3:33065/ using document root <none> and password file <none>
I20260812 17:48:34.915961 23123 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:34.916036 23123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:34.916298 23123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:34.918617 23123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "e600db37beac4b86b0d1b76a3c1c094e"
format_stamp: "Formatted at 2026-08-12 17:48:34 on dist-test-slave-sb2z"
server_key: "267a3f8d882b413b1c9da6b885e13b60"
server_key_iv: "9529b5ac0cf87fc6350c0cf4e8e57183"
server_key_version: "encryptionkey@0"
I20260812 17:48:34.919263 23123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "e600db37beac4b86b0d1b76a3c1c094e"
format_stamp: "Formatted at 2026-08-12 17:48:34 on dist-test-slave-sb2z"
server_key: "267a3f8d882b413b1c9da6b885e13b60"
server_key_iv: "9529b5ac0cf87fc6350c0cf4e8e57183"
server_key_version: "encryptionkey@0"
I20260812 17:48:34.923681 23123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 17:48:34.926582 23147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:34.928258 23123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:34.928434 23123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "e600db37beac4b86b0d1b76a3c1c094e"
format_stamp: "Formatted at 2026-08-12 17:48:34 on dist-test-slave-sb2z"
server_key: "267a3f8d882b413b1c9da6b885e13b60"
server_key_iv: "9529b5ac0cf87fc6350c0cf4e8e57183"
server_key_version: "encryptionkey@0"
I20260812 17:48:34.928576 23123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:34.947351 23123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:34.950969 23123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:34.951222 23123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:34.952015 23123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:34.952948 23123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:34.953032 23123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:34.953106 23123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:34.953161 23123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:34.962702 23123 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:40171
I20260812 17:48:34.962724 23351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:40171 every 8 connection(s)
I20260812 17:48:34.963824 23123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:34.966367 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 23123
I20260812 17:48:34.966514 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:34.966856 13464 external_mini_cluster.cc:1428] Setting key 0c5015a7a2016b1136b78c92afcb114a
Aug 12 17:48:34 dist-test-slave-sb2z krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:34.979604 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:34.965847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:40559 (local address 127.13.38.62:37903)
0812 17:48:34.966005 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:34.966009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:34.967211 (+ 1202us) server_negotiation.cc:407] Connection header received
0812 17:48:34.968950 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:34.968953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:34.969012 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:34.969176 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:34.970453 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.970990 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.972311 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.972549 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.972664 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:34.975482 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:34.975485 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:34.975487 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:34.975571 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:34.977300 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:34.977909 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:34.977913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:34.977915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:34.978009 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:34.978382 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:34.978386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:34.978388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:34.978593 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:34.978708 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:34.979260 (+ 552us) server_negotiation.cc:299] Negotiation successful
0812 17:48:34.979395 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 17:48:34.980744 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:34.966295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:37903 (local address 127.13.38.3:40559)
0812 17:48:34.967011 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:34.967078 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:34.968725 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:34.969374 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:34.969381 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:34.969775 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:34.970288 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:34.970303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.971131 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.971140 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:34.972161 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:34.972172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:34.972730 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:34.972735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:34.972914 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:34.973594 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:34.973612 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:34.975264 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:34.977465 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:34.977472 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:34.977492 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:34.977783 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:34.978112 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:34.978116 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:34.978118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:34.978275 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:34.978709 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:34.978719 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:34.979004 (+ 285us) client_negotiation.cc:769] Sending connection context
0812 17:48:34.979218 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 17:48:34.979630 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":191,"threads_started":1}
I20260812 17:48:34.982275 23352 heartbeater.cc:347] Connected to a master server at 127.13.38.62:37903
I20260812 17:48:34.982617 23352 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:34.983220 23352 heartbeater.cc:511] Master 127.13.38.62:37903 requested a full tablet report, sending...
I20260812 17:48:34.984707 22611 ts_manager.cc:194] Registered new tserver with Master: e600db37beac4b86b0d1b76a3c1c094e (127.13.38.3:40171)
I20260812 17:48:34.985327 22611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:40559
I20260812 17:48:34.994020 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/test-admin.keytab.
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:35.016403 13464 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 17:48:35.019807 23360 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:35.018152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48370 (local address 127.13.38.1:46531)
0812 17:48:35.018485 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:35.018492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:35.018518 (+ 26us) server_negotiation.cc:407] Connection header received
0812 17:48:35.018576 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:35.018583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:35.018787 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:35.018939 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:35.019643 (+ 704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48370: BlockingRecv error: recv got EOF from 127.0.0.1:48370 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":119,"threads_started":1}
W20260812 17:48:35.020192 13464 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/krb5cc' of type FILE
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:35.021341 13464 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:35.022305 13464 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.1@KRBTEST.COM
I20260812 17:48:35.033102 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:35.023272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.13.38.1:46531)
0812 17:48:35.023403 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:35.023409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:35.023504 (+ 95us) server_negotiation.cc:407] Connection header received
0812 17:48:35.023708 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:35.023711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:35.023757 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:35.023844 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:35.024402 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:35.025205 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:35.026055 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:35.026449 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:35.026571 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:35.029149 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:35.029161 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:35.029177 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:35.029204 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:35.031150 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:35.031727 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:35.031733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:35.031734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:35.031828 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:35.032121 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:35.032124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:35.032125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:35.032380 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:35.032539 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:35.032818 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 17:48:35.032974 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 17:48:35.037737 23360 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:35.036432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48384 (local address 127.13.38.1:46531)
0812 17:48:35.036577 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:35.036581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:35.036748 (+ 167us) server_negotiation.cc:407] Connection header received
0812 17:48:35.036850 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:35.036852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:35.036991 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:35.037074 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:35.037596 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48384: BlockingRecv error: recv got EOF from 127.0.0.1:48384 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 17:48:35.037735 13464 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/krb5cc' of type FILE
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:35.038973 13464 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:35.040126 13464 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.1@KRBTEST.COM
I20260812 17:48:35.049894 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:35.041046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.13.38.1:46531)
0812 17:48:35.041246 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:35.041251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:35.041390 (+ 139us) server_negotiation.cc:407] Connection header received
0812 17:48:35.041585 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:35.041588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:35.041647 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:35.041746 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:35.042342 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:35.042905 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:35.043891 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:35.044145 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:35.044276 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:35.046731 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:35.046734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:35.046736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:35.046759 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:35.048314 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:35.048768 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:35.048772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:35.048773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:35.048851 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:35.049168 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:35.049170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:35.049171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:35.049335 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:35.049453 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:35.049673 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 17:48:35.049774 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260812 17:48:35.052709 23360 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:35.051858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48406 (local address 127.13.38.1:46531)
0812 17:48:35.051996 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:35.051999 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:35.052027 (+ 28us) server_negotiation.cc:407] Connection header received
0812 17:48:35.052144 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:35.052146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:35.052181 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:35.052244 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:35.052620 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48406: BlockingRecv error: recv got EOF from 127.0.0.1:48406 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 17:48:35.052901 13464 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786556880371321-13464-0/krb5kdc/krb5cc' of type FILE
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:35.053954 13464 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:35.054949 13464 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.1@KRBTEST.COM
I20260812 17:48:35.063098 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:35.055907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.13.38.1:46531)
0812 17:48:35.056034 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:35.056037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:35.056050 (+ 13us) server_negotiation.cc:407] Connection header received
0812 17:48:35.056155 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:35.056157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:35.056191 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:35.056268 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:35.056727 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:35.057229 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:35.057970 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:35.058171 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:35.058246 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:35.060510 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:35.060513 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:35.060515 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:35.060537 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:35.061757 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:35.062144 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:35.062146 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:35.062147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:35.062218 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:35.062448 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:35.062450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:35.062451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:35.062603 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:35.062725 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:35.062895 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 17:48:35.062990 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 17:48:35.064235 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 22706
I20260812 17:48:35.076651 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 22916
I20260812 17:48:35.087855 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 23123
I20260812 17:48:35.110849 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 22561
2026-08-12T17:48:35Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2555 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[23380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[23380](info): set up 2 sockets
Aug 12 17:48:35 dist-test-slave-sb2z krb5kdc[23380](info): commencing operation
krb5kdc: starting...
W20260812 17:48:36.401674 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:36 dist-test-slave-sb2z krb5kdc[23380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.62.keytab.
Entry for principal oryx/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.62.keytab.
I20260812 17:48:36.547370 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:38917
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.62.keytab
--principal=oryx/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:38917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:36.667918 23397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:36.668295 23397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:36.668376 23397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:36.672675 23397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:36.672791 23397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:36.672827 23397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:36.672871 23397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:36.672907 23397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:36.678588 23397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:38917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.62.keytab
--principal=oryx/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:38917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:36.679934 23397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:36.681259 23397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:36.690763 23409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:36.690763 23413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:36.690766 23408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:36.691227 23397 server_base.cc:1034] running on GCE node
I20260812 17:48:36.691954 23397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:36.693296 23397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:36.694523 23397 hybrid_clock.cc:648] HybridClock initialized: now 1786556916694498 us; error 40 us; skew 500 ppm
Aug 12 17:48:36 dist-test-slave-sb2z krb5kdc[23380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556916, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:36.697669 23397 init.cc:377] Logged in from keytab as oryx/127.13.38.62@KRBTEST.COM (short username oryx)
I20260812 17:48:36.698746 23397 webserver.cc:460] Webserver started at http://127.13.38.62:38507/ using document root <none> and password file <none>
I20260812 17:48:36.699333 23397 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:36.699400 23397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:36.699659 23397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:36.701941 23397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "0104f86e900746688510a9947d247481"
format_stamp: "Formatted at 2026-08-12 17:48:36 on dist-test-slave-sb2z"
server_key: "ad0a01e42d8fa82dbc23fb4c6a007dcb"
server_key_iv: "3c518b57be6cc5d3380ae3ebeea8bcd8"
server_key_version: "encryptionkey@0"
I20260812 17:48:36.702494 23397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "0104f86e900746688510a9947d247481"
format_stamp: "Formatted at 2026-08-12 17:48:36 on dist-test-slave-sb2z"
server_key: "ad0a01e42d8fa82dbc23fb4c6a007dcb"
server_key_iv: "3c518b57be6cc5d3380ae3ebeea8bcd8"
server_key_version: "encryptionkey@0"
I20260812 17:48:36.706840 23397 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:48:36.709834 23427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:36.711280 23397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:36.711453 23397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "0104f86e900746688510a9947d247481"
format_stamp: "Formatted at 2026-08-12 17:48:36 on dist-test-slave-sb2z"
server_key: "ad0a01e42d8fa82dbc23fb4c6a007dcb"
server_key_iv: "3c518b57be6cc5d3380ae3ebeea8bcd8"
server_key_version: "encryptionkey@0"
I20260812 17:48:36.711647 23397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:36.739791 23397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:36.744097 23397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:36.744323 23397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:36.753079 23397 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:38917
I20260812 17:48:36.753434 23510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:38917 every 8 connection(s)
I20260812 17:48:36.754323 23397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:36.756294 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 23397
I20260812 17:48:36.756471 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:36.756790 13464 external_mini_cluster.cc:1428] Setting key 87202bce07a582079609d166402a57e1
I20260812 17:48:36.757872 23511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:36.765412 23511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481: Bootstrap starting.
Aug 12 17:48:36 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:36.768196 23511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:36.769279 23511 log.cc:824] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:36.771941 23511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481: No bootstrap required, opened a new log
I20260812 17:48:36.774257 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:36.758448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.13.38.62:38917)
0812 17:48:36.758932 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:36.758954 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:36.758999 (+ 45us) server_negotiation.cc:407] Connection header received
0812 17:48:36.759947 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:36.759974 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:36.760447 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:36.760870 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:36.761609 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:36.763119 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:36.764257 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:36.764975 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:36.765164 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:36.768200 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:36.768209 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:36.768232 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:36.768274 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:36.770312 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:36.771118 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:36.771124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:36.771130 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:36.771279 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:36.771692 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:36.771696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:36.771697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:36.772284 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:36.772491 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:36.772843 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 17:48:36.773197 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":133,"threads_started":1}
I20260812 17:48:36.776712 23511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } }
I20260812 17:48:36.777114 23511 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:36.777204 23511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0104f86e900746688510a9947d247481, State: Initialized, Role: FOLLOWER
I20260812 17:48:36.777657 23511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } }
I20260812 17:48:36.777822 23511 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:36.777895 23511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:36.777999 23511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:36.779042 23511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } }
I20260812 17:48:36.779424 23511 leader_election.cc:302] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0104f86e900746688510a9947d247481; no voters:
I20260812 17:48:36.779906 23511 leader_election.cc:288] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:36.780061 23518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:36.780367 23518 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 1 LEADER]: Becoming Leader. State: Replica: 0104f86e900746688510a9947d247481, State: Running, Role: LEADER
I20260812 17:48:36.780819 23518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } }
I20260812 17:48:36.781386 23511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:36.782728 23523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0104f86e900746688510a9947d247481. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } }
I20260812 17:48:36.782877 23523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:36.783239 23522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } }
I20260812 17:48:36.783325 23522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:36.783357 23531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:36.786913 23531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:36.793649 23531 catalog_manager.cc:1347] Generated new cluster ID: dfa9bbf061bc403f8452108c10365386
I20260812 17:48:36.793751 23531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:36.804826 23531 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:36.806241 23531 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:36.813021 23531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481: Generated new TSK 0
I20260812 17:48:36.813954 23531 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab.
Entry for principal oryx/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab.
I20260812 17:48:36.851897 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:38917
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab
--principal=oryx/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:36.974378 23547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:36.974720 23547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:36.974804 23547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:36.979238 23547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:36.979365 23547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:36.979480 23547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:36.985862 23547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab
--principal=oryx/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:38917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:36.987131 23547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:36.988327 23547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:36.999313 23556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:36.999320 23564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:36.999303 23557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:36.999696 23547 server_base.cc:1034] running on GCE node
I20260812 17:48:37.000336 23547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:37.001153 23547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:37.002421 23547 hybrid_clock.cc:648] HybridClock initialized: now 1786556917002373 us; error 57 us; skew 500 ppm
Aug 12 17:48:37 dist-test-slave-sb2z krb5kdc[23380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556917, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:37.005781 23547 init.cc:377] Logged in from keytab as oryx/127.13.38.1@KRBTEST.COM (short username oryx)
I20260812 17:48:37.006843 23547 webserver.cc:460] Webserver started at http://127.13.38.1:36315/ using document root <none> and password file <none>
I20260812 17:48:37.007519 23547 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:37.007681 23547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:37.007941 23547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:37.010172 23547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "e25783f400113d4fc8bc7e08de1de92d"
server_key_iv: "a0f1858f36c57f783bfa7ced76800d00"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.010814 23547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "e25783f400113d4fc8bc7e08de1de92d"
server_key_iv: "a0f1858f36c57f783bfa7ced76800d00"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.015110 23547 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:48:37.018083 23577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:37.019747 23547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 17:48:37.019914 23547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "e25783f400113d4fc8bc7e08de1de92d"
server_key_iv: "a0f1858f36c57f783bfa7ced76800d00"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.020067 23547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:37.050127 23547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:37.053607 23547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:37.053848 23547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:37.054517 23547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:37.055404 23547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:37.055480 23547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:37.055576 23547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:37.055634 23547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:37.064980 23547 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:45161
I20260812 17:48:37.065004 23754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:45161 every 8 connection(s)
I20260812 17:48:37.065917 23547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:37.071334 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 23547
I20260812 17:48:37.071442 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:37.071774 13464 external_mini_cluster.cc:1428] Setting key c87da9de2a3b1765e2965422f437c307
Aug 12 17:48:37 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556917, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:37.082152 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.067933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:41535 (local address 127.13.38.62:38917)
0812 17:48:37.068081 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:37.068086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:37.069006 (+ 920us) server_negotiation.cc:407] Connection header received
0812 17:48:37.070485 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:37.070488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:37.070550 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:37.070640 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:37.072087 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.072809 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.074606 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.074836 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.074924 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.077762 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:37.077769 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:37.077775 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:37.077811 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:37.079729 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.080273 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.080279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.080284 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.080393 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.080774 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.080780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.080785 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.081069 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:37.081178 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:37.081776 (+ 598us) server_negotiation.cc:299] Negotiation successful
0812 17:48:37.081941 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 17:48:37.083014 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.068323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.1:41535)
0812 17:48:37.068821 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:37.068868 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:37.070238 (+ 1370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:37.070822 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:37.070830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:37.071231 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:37.071925 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.071942 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.072952 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.072955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.074455 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.074471 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.075010 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.075017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.075266 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.076064 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:37.076082 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:37.077569 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:37.079861 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.079867 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.079880 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.080160 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.080501 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.080504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.080505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.080650 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.081185 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:37.081190 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:37.081505 (+ 315us) client_negotiation.cc:769] Sending connection context
0812 17:48:37.081685 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 17:48:37.082097 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":133,"threads_started":1}
WARNING: no policy specified for oryx/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:48:37.084367 23755 heartbeater.cc:347] Connected to a master server at 127.13.38.62:38917
I20260812 17:48:37.084643 23755 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:37.085263 23755 heartbeater.cc:511] Master 127.13.38.62:38917 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.2@KRBTEST.COM" created.
I20260812 17:48:37.087018 23452 ts_manager.cc:194] Registered new tserver with Master: b8e8d5ffa8c347f2a3ad8bc1e448d811 (127.13.38.1:45161)
I20260812 17:48:37.088393 23452 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:41535
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab.
Entry for principal oryx/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab.
I20260812 17:48:37.107244 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:38917
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab
--principal=oryx/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:37.228072 23767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:37.228389 23767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:37.228434 23767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:37.232481 23767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:37.232549 23767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:37.232609 23767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:37.237964 23767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab
--principal=oryx/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:38917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:37.239053 23767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:37.240083 23767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:37.250216 23776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:37.250176 23778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:37.250176 23782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:37.250481 23767 server_base.cc:1034] running on GCE node
I20260812 17:48:37.250957 23767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:37.251749 23767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:37.252938 23767 hybrid_clock.cc:648] HybridClock initialized: now 1786556917252918 us; error 33 us; skew 500 ppm
Aug 12 17:48:37 dist-test-slave-sb2z krb5kdc[23380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556917, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:37.255857 23767 init.cc:377] Logged in from keytab as oryx/127.13.38.2@KRBTEST.COM (short username oryx)
I20260812 17:48:37.256860 23767 webserver.cc:460] Webserver started at http://127.13.38.2:44881/ using document root <none> and password file <none>
I20260812 17:48:37.257474 23767 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:37.257572 23767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:37.257819 23767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:37.260052 23767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "f15015c53e544edeb79ea507c2cd2b18"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "1235e64cd91265e2bcb4879a09eb7027"
server_key_iv: "ededee90916c20356226321063158238"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.260679 23767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "f15015c53e544edeb79ea507c2cd2b18"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "1235e64cd91265e2bcb4879a09eb7027"
server_key_iv: "ededee90916c20356226321063158238"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.264667 23767 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:48:37.267216 23792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:37.268795 23767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:37.268989 23767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "f15015c53e544edeb79ea507c2cd2b18"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "1235e64cd91265e2bcb4879a09eb7027"
server_key_iv: "ededee90916c20356226321063158238"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.269134 23767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:37.285106 23767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:37.289333 23767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:37.289602 23767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:37.290251 23767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:37.291204 23767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:37.291280 23767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:37.291350 23767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:37.291383 23767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:37.300922 23767 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:35863
I20260812 17:48:37.300983 23963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:35863 every 8 connection(s)
I20260812 17:48:37.302232 23767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:37.305790 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 23767
I20260812 17:48:37.305894 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:37.306182 13464 external_mini_cluster.cc:1428] Setting key 381fcc66f3384fc8969eadb023c15a0d
Aug 12 17:48:37 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556917, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:48:37.318764 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.304259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:37667 (local address 127.13.38.62:38917)
0812 17:48:37.304415 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:37.304419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:37.305350 (+ 931us) server_negotiation.cc:407] Connection header received
0812 17:48:37.306952 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:37.306956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:37.307013 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:37.307117 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:37.308502 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.309072 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.310675 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.310904 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.310992 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.314168 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:37.314171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:37.314173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:37.314200 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:37.315895 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.316685 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.316692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.316695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.316799 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.317267 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.317274 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.317276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.317595 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:37.317716 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:37.318458 (+ 742us) server_negotiation.cc:299] Negotiation successful
0812 17:48:37.318619 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 17:48:37.320194 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.304670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.2:37667)
0812 17:48:37.305179 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:37.305230 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:37.306667 (+ 1437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:37.307283 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:37.307289 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:37.307834 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:37.308339 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.308354 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.309214 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.309220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.310524 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.310540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.311083 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.311091 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.311330 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.312115 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:37.312133 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:37.313946 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:37.316020 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.316029 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.316050 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.316513 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.316932 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.316935 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.316937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.317129 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.317734 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:37.317761 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:37.318117 (+ 356us) client_negotiation.cc:769] Sending connection context
0812 17:48:37.318970 (+ 853us) client_negotiation.cc:240] Negotiation successful
0812 17:48:37.319269 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":130,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.3@KRBTEST.COM" created.
I20260812 17:48:37.321614 23964 heartbeater.cc:347] Connected to a master server at 127.13.38.62:38917
I20260812 17:48:37.321904 23964 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:37.322613 23964 heartbeater.cc:511] Master 127.13.38.62:38917 requested a full tablet report, sending...
I20260812 17:48:37.325038 23452 ts_manager.cc:194] Registered new tserver with Master: f15015c53e544edeb79ea507c2cd2b18 (127.13.38.2:35863)
I20260812 17:48:37.326103 23452 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:37667
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab.
Entry for principal oryx/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab.
I20260812 17:48:37.340608 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:38917
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab
--principal=oryx/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:37.461266 23975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:37.461571 23975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:37.461618 23975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:37.465833 23975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:37.465917 23975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:37.465978 23975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:37.471704 23975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab
--principal=oryx/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:38917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:37.473001 23975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:37.474182 23975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:37.484897 23985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:37.484897 23981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:37.484921 23983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:37.485211 23975 server_base.cc:1034] running on GCE node
I20260812 17:48:37.485749 23975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:37.486502 23975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:37.487712 23975 hybrid_clock.cc:648] HybridClock initialized: now 1786556917487691 us; error 30 us; skew 500 ppm
Aug 12 17:48:37 dist-test-slave-sb2z krb5kdc[23380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556917, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:37.490914 23975 init.cc:377] Logged in from keytab as oryx/127.13.38.3@KRBTEST.COM (short username oryx)
I20260812 17:48:37.492008 23975 webserver.cc:460] Webserver started at http://127.13.38.3:39913/ using document root <none> and password file <none>
I20260812 17:48:37.492674 23975 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:37.492745 23975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:37.492977 23975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:37.495222 23975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "1250825c07de44a2af1c6a4a1487de2c"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "b396f568d44d53449f18e077356ac8b0"
server_key_iv: "e5f5fc5a8f317506911ee204328ae93b"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.495940 23975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "1250825c07de44a2af1c6a4a1487de2c"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "b396f568d44d53449f18e077356ac8b0"
server_key_iv: "e5f5fc5a8f317506911ee204328ae93b"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.500047 23975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:48:37.502671 23997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:37.504192 23975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:37.504349 23975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "1250825c07de44a2af1c6a4a1487de2c"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "b396f568d44d53449f18e077356ac8b0"
server_key_iv: "e5f5fc5a8f317506911ee204328ae93b"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.504478 23975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:37.515424 23975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:37.519047 23975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:37.519282 23975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:37.519987 23975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:37.520915 23975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:37.520991 23975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:37.521075 23975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:37.521123 23975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:37.530488 23975 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:32929
I20260812 17:48:37.530543 24168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:32929 every 8 connection(s)
I20260812 17:48:37.531510 23975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:37.538919 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 23975
I20260812 17:48:37.539047 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:37.539397 13464 external_mini_cluster.cc:1428] Setting key 99bcdf42fe67796eb532ca5d1f40e29a
Aug 12 17:48:37 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556917, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:37.547246 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.533650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:40901 (local address 127.13.38.62:38917)
0812 17:48:37.533847 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:37.533851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:37.534762 (+ 911us) server_negotiation.cc:407] Connection header received
0812 17:48:37.536378 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:37.536382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:37.536453 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:37.536567 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:37.537865 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.538714 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.540322 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.540612 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.540738 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.543491 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:37.543494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:37.543496 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:37.543607 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:37.545177 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.545687 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.545690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.545691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.545769 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.546102 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.546104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.546105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.546289 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:37.546364 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:37.546977 (+ 613us) server_negotiation.cc:299] Negotiation successful
0812 17:48:37.547105 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 17:48:37.548105 24172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.534017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.3:40901)
0812 17:48:37.534582 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:37.534631 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:37.536136 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:37.536758 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:37.536765 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:37.537169 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:37.537696 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.537711 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.538870 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.538873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.540146 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.540158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.540785 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.540788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.541032 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.541720 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:37.541744 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:37.543316 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:37.545292 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.545297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.545312 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.545594 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.545850 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.545853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.545854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.545998 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.546380 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:37.546385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:37.546653 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 17:48:37.546825 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 17:48:37.547115 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":139,"threads_started":1}
I20260812 17:48:37.549505 24170 heartbeater.cc:347] Connected to a master server at 127.13.38.62:38917
I20260812 17:48:37.549909 24170 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:37.550549 24170 heartbeater.cc:511] Master 127.13.38.62:38917 requested a full tablet report, sending...
I20260812 17:48:37.551795 23452 ts_manager.cc:194] Registered new tserver with Master: 1250825c07de44a2af1c6a4a1487de2c (127.13.38.3:32929)
I20260812 17:48:37.552448 23452 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:40901
I20260812 17:48:37.553999 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.38.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.61.keytab.
Entry for principal oryx/127.13.38.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.61.keytab.
I20260812 17:48:37.586412 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.61:41603
--webserver_interface=127.13.38.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.61.keytab
--principal=oryx/127.13.38.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:38917,127.13.38.61:41603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:37.704638 24176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:37.704984 24176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:37.705093 24176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:37.709388 24176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:37.709488 24176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:37.709524 24176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:37.709575 24176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:37.709617 24176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:37.715178 24176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:38917,127.13.38.61:41603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.61.keytab
--principal=oryx/127.13.38.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.61:41603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.38.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
W20260812 17:48:37.715473 24176 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.38.62:38917,127.13.38.61:41603'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 17:48:37.716552 24176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:37.717641 24176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:37.727231 24188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:37.727231 24184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:37.727324 24185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:37.727370 24176 server_base.cc:1034] running on GCE node
I20260812 17:48:37.728336 24176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:37.729650 24176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:37.730858 24176 hybrid_clock.cc:648] HybridClock initialized: now 1786556917730842 us; error 34 us; skew 500 ppm
Aug 12 17:48:37 dist-test-slave-sb2z krb5kdc[23380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556917, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:37.737812 24176 init.cc:377] Logged in from keytab as oryx/127.13.38.61@KRBTEST.COM (short username oryx)
I20260812 17:48:37.738929 24176 webserver.cc:460] Webserver started at http://127.13.38.61:33585/ using document root <none> and password file <none>
I20260812 17:48:37.739660 24176 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:37.739766 24176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:37.740033 24176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:37.742288 24176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data/instance:
uuid: "df1d07a897684999acc8fda17f1b348e"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "3b83b2285698ab72a98166df177285cf"
server_key_iv: "595338af7546c974d34e80299ab39d1d"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.742935 24176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/wal/instance:
uuid: "df1d07a897684999acc8fda17f1b348e"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "3b83b2285698ab72a98166df177285cf"
server_key_iv: "595338af7546c974d34e80299ab39d1d"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.747303 24176 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 17:48:37.750288 24201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:37.751884 24176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:37.752069 24176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/wal
uuid: "df1d07a897684999acc8fda17f1b348e"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "3b83b2285698ab72a98166df177285cf"
server_key_iv: "595338af7546c974d34e80299ab39d1d"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.752205 24176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:37.769021 24176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:37.772253 24176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:37.772501 24176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:37.780623 24176 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.61:41603
I20260812 17:48:37.780938 24289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.61:41603 every 8 connection(s)
I20260812 17:48:37.781853 24176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
I20260812 17:48:37.784190 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 24176
I20260812 17:48:37.785593 24290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:37.790539 24290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 17:48:37 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.38.61@KRBTEST.COM
Aug 12 17:48:37 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556917, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.61@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:37.800662 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.785903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33788 (local address 127.13.38.61:41603)
0812 17:48:37.786401 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:37.786422 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:37.786466 (+ 44us) server_negotiation.cc:407] Connection header received
0812 17:48:37.787363 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:37.787388 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:37.787866 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:37.788259 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:37.789005 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.790520 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.791680 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.792352 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.792485 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.795607 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:37.795615 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:37.795625 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:37.795654 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:37.797587 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.798089 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.798094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.798099 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.798211 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.798522 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.798526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.798528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.799007 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:37.799246 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:37.799575 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 17:48:37.799867 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":184,"threads_started":1}
I20260812 17:48:37.802757 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.792700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33368)
0812 17:48:37.793224 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:37.793269 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:37.793422 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:37.793752 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:37.793759 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:37.794147 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:37.794332 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.794341 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.795223 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.795230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.796200 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.796211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.796871 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.796874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.796961 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.797466 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:37.797478 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:37.799033 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:37.800992 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.801003 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.801010 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.801249 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.801548 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.801551 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.801553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.801649 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.802111 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:37.802118 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:37.802238 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 17:48:37.802385 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 17:48:37.802578 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":192,"threads_started":1}
I20260812 17:48:37.802739 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.792756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33368 (local address 127.13.38.62:38917)
0812 17:48:37.792911 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:37.792917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:37.793362 (+ 445us) server_negotiation.cc:407] Connection header received
0812 17:48:37.793536 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:37.793539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:37.793600 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:37.793695 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:37.794475 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.795074 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.796329 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.796712 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.796849 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.799195 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:37.799199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:37.799201 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:37.799226 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:37.800864 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.801359 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.801363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.801365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.801448 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.801747 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.801751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.801752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.801979 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:37.802111 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:37.802442 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 17:48:37.802567 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 17:48:37.804189 24290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 17:48:37 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556917, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.61@KRBTEST.COM for oryx/127.13.38.61@KRBTEST.COM
I20260812 17:48:37.813076 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.804825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:41603 (local address 127.0.0.1:33798)
0812 17:48:37.805041 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:37.805068 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:37.805167 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:37.805452 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:37.805455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:37.805634 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:37.805717 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.805722 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.806804 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.806807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.807333 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.807340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.807876 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.807880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.808018 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.808474 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:37.808486 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:37.809813 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:37.811513 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.811518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.811519 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.811750 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.812044 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.812047 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.812049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.812130 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.812520 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:37.812524 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:37.812613 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 17:48:37.812720 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 17:48:37.812870 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":1}
I20260812 17:48:37.813076 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.804944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33798 (local address 127.13.38.61:41603)
0812 17:48:37.805077 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:37.805083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:37.805121 (+ 38us) server_negotiation.cc:407] Connection header received
0812 17:48:37.805279 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:37.805282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:37.805326 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:37.805442 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:37.805828 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.806677 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.807466 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.807753 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.807866 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.809958 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:37.809962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:37.809963 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:37.809986 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:37.811334 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.811853 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.811859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.811860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.811942 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.812220 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.812224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.812226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.812413 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:37.812519 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:37.812768 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 17:48:37.812925 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":72}
I20260812 17:48:37.816846 24290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: Bootstrap starting.
I20260812 17:48:37.819393 24290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:37.820256 24290 log.cc:824] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:37.822451 24290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: No bootstrap required, opened a new log
I20260812 17:48:37.826002 24290 raft_consensus.cc:348] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } }
I20260812 17:48:37.826231 24290 raft_consensus.cc:374] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:37.826282 24290 raft_consensus.cc:729] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1d07a897684999acc8fda17f1b348e, State: Initialized, Role: FOLLOWER
I20260812 17:48:37.826759 24290 consensus_queue.cc:260] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } }
I20260812 17:48:37.827291 24298 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } } }
I20260812 17:48:37.827462 24298 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:48:37.828136 24290 sys_catalog.cc:564] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: configured and running, proceeding with master startup.
W20260812 17:48:37.833364 24318 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:48:37.833465 24318 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:48:37.836140 24176 master_runner.cc:428] Detected that this master df1d07a897684999acc8fda17f1b348e is joining an existing cluster
I20260812 17:48:37.836210 24176 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.38.61:41603
I20260812 17:48:37.848093 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.838680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33374 (local address 127.13.38.62:38917)
0812 17:48:37.838832 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:37.838839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:37.839169 (+ 330us) server_negotiation.cc:407] Connection header received
0812 17:48:37.839218 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:37.839223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:37.839275 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:37.839356 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:37.840099 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.840936 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.842258 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.842605 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.842685 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.844298 (+ 1613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:37.844302 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:37.844305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:37.844336 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:37.846195 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.846679 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.846683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.846685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.846777 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.847072 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.847075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.847077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.847304 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:37.847375 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:37.847794 (+ 419us) server_negotiation.cc:299] Negotiation successful
0812 17:48:37.847917 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 17:48:37.848088 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.839081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33802 (local address 127.13.38.61:41603)
0812 17:48:37.839203 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:37.839208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:37.839556 (+ 348us) server_negotiation.cc:407] Connection header received
0812 17:48:37.839653 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:37.839656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:37.839698 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:37.839778 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:37.840329 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.840876 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.841764 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.842048 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.842168 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.844183 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:37.844187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:37.844190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:37.844220 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:37.846078 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.846633 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.846640 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.846644 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.846732 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.847081 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.847085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.847087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.847313 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:37.847375 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:37.847756 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 17:48:37.847950 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":53}
I20260812 17:48:37.848104 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.839044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:41603 (local address 127.0.0.1:33802)
0812 17:48:37.839414 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:37.839428 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:37.839564 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:37.839838 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:37.839841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:37.840092 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:37.840212 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.840218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.841021 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.841025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.841640 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.841647 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.842172 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.842176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.842270 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.843471 (+ 1201us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:37.843508 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:37.844035 (+ 527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:37.846211 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.846215 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.846217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.846520 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.846842 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.846848 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.846850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.846963 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.847424 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:37.847426 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:37.847510 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 17:48:37.847704 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 17:48:37.847875 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"spinlock_wait_cycles":5888,"thread_start_us":88,"threads_started":1}
I20260812 17:48:37.848879 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.838519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33374)
0812 17:48:37.838872 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:37.838888 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:37.839041 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:37.839407 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:37.839410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:37.839728 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:37.839939 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.839952 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.841080 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.841098 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.842115 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.842126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.842842 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.842847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.842952 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.843545 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:37.843565 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:37.844151 (+ 586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:37.846286 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.846292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.846295 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.846561 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.846884 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.846887 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.846889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.846973 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.847449 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:37.847452 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:37.847588 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 17:48:37.848540 (+ 952us) client_negotiation.cc:240] Negotiation successful
0812 17:48:37.848690 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":32,"spinlock_wait_cycles":6400,"thread_start_us":119,"threads_started":1}
Aug 12 17:48:37 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556916, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.62@KRBTEST.COM for oryx/127.13.38.61@KRBTEST.COM
WARNING: no policy specified for oryx/127.13.38.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.60@KRBTEST.COM" created.
I20260812 17:48:37.868762 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.853912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:41603 (local address 127.0.0.1:33804)
0812 17:48:37.857406 (+ 3494us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:37.857486 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:37.857653 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:37.858028 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:37.858037 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:37.858600 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:37.858820 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.858832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.859979 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.859987 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.861176 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.861194 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.861816 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.861821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.861976 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.862665 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:37.862686 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:37.864642 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:37.866648 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.866660 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.866669 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.866998 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.867280 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.867283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.867285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.867390 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.867972 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:37.867980 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:37.868167 (+ 187us) client_negotiation.cc:769] Sending connection context
0812 17:48:37.868339 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 17:48:37.868560 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3267,"thread_start_us":105,"threads_started":1}
I20260812 17:48:37.869246 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.853932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33804 (local address 127.13.38.61:41603)
0812 17:48:37.854064 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:37.854068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:37.857580 (+ 3512us) server_negotiation.cc:407] Connection header received
0812 17:48:37.857822 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:37.857826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:37.857881 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:37.857962 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:37.858955 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.859820 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.861334 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.861674 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.861776 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.864822 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:37.864827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:37.864830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:37.864861 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:37.866523 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.867104 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.867107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.867109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.867188 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.867516 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.867519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.867520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.867834 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:37.867951 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:37.868532 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 17:48:37.868753 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 17:48:37.870846 23451 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.38.61:41603: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "df1d07a897684999acc8fda17f1b348e"
member_type: NON_VOTER
last_known_addr {
host: "127.13.38.61"
port: 41603
}
attrs {
promote: true
}
}
dest_uuid: "0104f86e900746688510a9947d247481"
cas_config_opid_index: -1
I20260812 17:48:37.871701 23451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true } }
I20260812 17:48:37.874435 24336 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER df1d07a897684999acc8fda17f1b348e (127.13.38.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true } } }
I20260812 17:48:37.875160 24249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:37.876961 24336 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.38.61:41603
I20260812 17:48:37.877197 24249 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 FOLLOWER]: Refusing update from remote peer 0104f86e900746688510a9947d247481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 17:48:37.877753 24176 master.cc:537] Master@127.13.38.61:41603 shutting down...
I20260812 17:48:37.878540 24336 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:37.879029 24334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true } } }
I20260812 17:48:37.879691 24334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: This master's current role is: LEADER
W20260812 17:48:37.882246 23430 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260812 17:48:37.883242 24176 raft_consensus.cc:2229] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:37.883596 24176 raft_consensus.cc:2258] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 17:48:37.883579 23430 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 -> Peer df1d07a897684999acc8fda17f1b348e (127.13.38.61:41603): Couldn't send request to peer df1d07a897684999acc8fda17f1b348e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 17:48:37.883715 24176 tablet_replica.cc:330] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.60.keytab.
Entry for principal oryx/127.13.38.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.60.keytab.
I20260812 17:48:37.892282 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.60:42247
--webserver_interface=127.13.38.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.60.keytab
--principal=oryx/127.13.38.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:38917,127.13.38.61:41603,127.13.38.60:42247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:48:37.899421 24176 master.cc:552] Master@127.13.38.61:41603 shutdown complete.
I20260812 17:48:37.899505 24176 master_runner.cc:305] Clearing existing system tablet
I20260812 17:48:37.899864 24176 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:37.899955 24176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:37.901785 24176 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 17:48:37.902938 24345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:37.903156 24176 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:37.903276 24176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:37.903311 24176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/wal
uuid: "df1d07a897684999acc8fda17f1b348e"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "3b83b2285698ab72a98166df177285cf"
server_key_iv: "595338af7546c974d34e80299ab39d1d"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.905771 24176 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:37.908425 24176 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 17:48:37.908527 24176 log.cc:1196] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 17:48:37.908809 24176 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: Deleting consensus metadata
I20260812 17:48:37.908989 24176 master_runner.cc:315] Copying system tablet from 127.13.38.62:38917
I20260812 17:48:37.910471 24176 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: tablet copy: Beginning tablet copy session from remote peer at address 127.13.38.62:38917
I20260812 17:48:37.921041 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.912041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33376 (local address 127.13.38.62:38917)
0812 17:48:37.912186 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:37.912191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:37.912459 (+ 268us) server_negotiation.cc:407] Connection header received
0812 17:48:37.912607 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:37.912610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:37.912669 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:37.912764 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:37.913316 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.914127 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.915416 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.915811 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.915939 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.917325 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:37.917329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:37.917333 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:37.917362 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:37.919125 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.919674 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.919678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.919680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.919773 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:37.920086 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:37.920089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:37.920091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:37.920331 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:37.920431 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:37.920732 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 17:48:37.920861 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 17:48:37.921052 24353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:37.911906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33376)
0812 17:48:37.912326 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:37.912342 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:37.912495 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:37.912796 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:37.912799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:37.913038 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:37.913189 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.913196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.914258 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.914262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.915287 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:37.915300 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:37.915960 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:37.915965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:37.916077 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:37.916687 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:37.916702 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:37.917174 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:37.919249 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.919253 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.919255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.919520 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.919875 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:37.919878 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:37.919880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:37.919988 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:37.920433 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:37.920436 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:37.920545 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 17:48:37.920658 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 17:48:37.920811 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":134,"threads_started":1}
I20260812 17:48:37.922194 23483 tablet_copy_service.cc:140] P 0104f86e900746688510a9947d247481: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer df1d07a897684999acc8fda17f1b348e ({username='oryx', principal='oryx/127.13.38.61@KRBTEST.COM'} at 127.0.0.1:33376)
I20260812 17:48:37.922406 23483 tablet_copy_service.cc:161] P 0104f86e900746688510a9947d247481: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer df1d07a897684999acc8fda17f1b348e at {username='oryx', principal='oryx/127.13.38.61@KRBTEST.COM'} at 127.0.0.1:33376: session id = df1d07a897684999acc8fda17f1b348e-00000000000000000000000000000000
I20260812 17:48:37.924407 23483 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 17:48:37.926462 24176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 17:48:37.929100 24176 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: tablet copy: Starting download of 0 data blocks...
I20260812 17:48:37.929311 24176 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: tablet copy: Starting download of 1 WAL segments...
I20260812 17:48:37.931424 24176 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 17:48:37.933619 23483 tablet_copy_service.cc:342] P 0104f86e900746688510a9947d247481: Request end of tablet copy session df1d07a897684999acc8fda17f1b348e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.38.61@KRBTEST.COM'} at 127.0.0.1:33376
I20260812 17:48:37.933775 23483 tablet_copy_service.cc:433] P 0104f86e900746688510a9947d247481: ending tablet copy session df1d07a897684999acc8fda17f1b348e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer df1d07a897684999acc8fda17f1b348e
I20260812 17:48:37.942198 24176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:37.944552 24354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:37.944623 24355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:37.944788 24357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:37.947747 24176 server_base.cc:1034] running on GCE node
I20260812 17:48:37.947981 24176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:37.948343 24176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:37.949514 24176 hybrid_clock.cc:648] HybridClock initialized: now 1786556917949487 us; error 47 us; skew 500 ppm
Aug 12 17:48:37 dist-test-slave-sb2z krb5kdc[23380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556917, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:37.951395 24176 init.cc:377] Logged in from keytab as oryx/127.13.38.61@KRBTEST.COM (short username oryx)
I20260812 17:48:37.952009 24176 webserver.cc:460] Webserver started at http://127.13.38.61:35799/ using document root <none> and password file <none>
I20260812 17:48:37.952216 24176 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:37.952257 24176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:37.953761 24176 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 17:48:37.954886 24370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:37.955241 24176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 17:48:37.955328 24176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/wal
uuid: "df1d07a897684999acc8fda17f1b348e"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "3b83b2285698ab72a98166df177285cf"
server_key_iv: "595338af7546c974d34e80299ab39d1d"
server_key_version: "encryptionkey@0"
I20260812 17:48:37.955415 24176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:37.967178 24176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:37.969681 24176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:37.976226 24176 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.61:41603
I20260812 17:48:37.976373 24450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.61:41603 every 8 connection(s)
I20260812 17:48:37.976912 24176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
I20260812 17:48:37.979143 24455 sys_catalog.cc:263] Verifying existing consensus state
I20260812 17:48:37.980556 24455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: Bootstrap starting.
I20260812 17:48:37.993856 24455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 17:48:37.994274 24455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: Bootstrap complete.
I20260812 17:48:37.994938 24455 raft_consensus.cc:348] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true } }
I20260812 17:48:37.995123 24455 raft_consensus.cc:729] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: df1d07a897684999acc8fda17f1b348e, State: Initialized, Role: LEARNER
I20260812 17:48:37.995263 24455 consensus_queue.cc:260] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true } }
I20260812 17:48:37.995709 24459 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true } } }
I20260812 17:48:37.995828 24455 sys_catalog.cc:564] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:37.995914 24459 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: This master's current role is: LEARNER
I20260812 17:48:38.000865 24478 catalog_manager.cc:1259] Loaded cluster ID: dfa9bbf061bc403f8452108c10365386
I20260812 17:48:38.000965 24478 catalog_manager.cc:1552] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: loading cluster ID for follower catalog manager: success
I20260812 17:48:38.003106 24478 catalog_manager.cc:1574] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: acquiring CA information for follower catalog manager: success
I20260812 17:48:38.004580 24478 catalog_manager.cc:1602] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 17:48:38.042390 24342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:38.042829 24342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:38.042909 24342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:38.047232 24342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:38.047358 24342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:38.047395 24342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:38.047430 24342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:38.047453 24342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:38.053110 24342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:38917,127.13.38.61:41603,127.13.38.60:42247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.60.keytab
--principal=oryx/127.13.38.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.60:42247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.38.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:38.054407 24342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:38.055498 24342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:38.064873 24487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:38.064880 24486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:38.064858 24490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:38.065349 24342 server_base.cc:1034] running on GCE node
I20260812 17:48:38.065971 24342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:38.067147 24342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:38.068342 24342 hybrid_clock.cc:648] HybridClock initialized: now 1786556918068291 us; error 71 us; skew 500 ppm
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:38.071332 24342 init.cc:377] Logged in from keytab as oryx/127.13.38.60@KRBTEST.COM (short username oryx)
I20260812 17:48:38.072363 24342 webserver.cc:460] Webserver started at http://127.13.38.60:34187/ using document root <none> and password file <none>
I20260812 17:48:38.073000 24342 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:38.073107 24342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:38.073365 24342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:38.075501 24342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data/instance:
uuid: "5b41fe185daa47f696e387ee7b9f1bd5"
format_stamp: "Formatted at 2026-08-12 17:48:38 on dist-test-slave-sb2z"
server_key: "5a68f52cda451873a26443adbb4d0f9e"
server_key_iv: "aac57aaebf33e4297eb4bf098023d565"
server_key_version: "encryptionkey@0"
I20260812 17:48:38.076159 24342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/wal/instance:
uuid: "5b41fe185daa47f696e387ee7b9f1bd5"
format_stamp: "Formatted at 2026-08-12 17:48:38 on dist-test-slave-sb2z"
server_key: "5a68f52cda451873a26443adbb4d0f9e"
server_key_iv: "aac57aaebf33e4297eb4bf098023d565"
server_key_version: "encryptionkey@0"
I20260812 17:48:38.080259 24342 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:48:38.083209 24504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:38.084635 24342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:48:38.084805 24342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/wal
uuid: "5b41fe185daa47f696e387ee7b9f1bd5"
format_stamp: "Formatted at 2026-08-12 17:48:38 on dist-test-slave-sb2z"
server_key: "5a68f52cda451873a26443adbb4d0f9e"
server_key_iv: "aac57aaebf33e4297eb4bf098023d565"
server_key_version: "encryptionkey@0"
I20260812 17:48:38.084931 24342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:38.091121 23755 heartbeater.cc:503] Master 127.13.38.62:38917 was elected leader, sending a full tablet report...
I20260812 17:48:38.100270 24342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:38.103811 24342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:38.104030 24342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:38.112876 24342 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.60:42247
I20260812 17:48:38.113216 24586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.60:42247 every 8 connection(s)
I20260812 17:48:38.113986 24342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
I20260812 17:48:38.117637 24587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:38.121668 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 24342
I20260812 17:48:38.122022 24587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.60@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.38.60@KRBTEST.COM
I20260812 17:48:38.136821 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.125046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33386 (local address 127.13.38.62:38917)
0812 17:48:38.125180 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.125187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.125405 (+ 218us) server_negotiation.cc:407] Connection header received
0812 17:48:38.125602 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.125605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.125657 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.125738 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.126822 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.127634 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.129163 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.129503 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.129641 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.132812 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.132818 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.132822 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.132862 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.134606 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.135183 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.135187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.135188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.135283 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.135685 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.135688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.135690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.135905 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.136025 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.136505 (+ 480us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.136643 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 17:48:38.137789 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.123395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37936 (local address 127.13.38.60:42247)
0812 17:48:38.123889 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.123911 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.123967 (+ 56us) server_negotiation.cc:407] Connection header received
0812 17:48:38.124851 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.124874 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.125326 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.125619 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.126567 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.127896 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.129137 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.129725 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.129894 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.133295 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.133301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.133313 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.133335 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.134817 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.135308 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.135314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.135315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.135424 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.135787 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.135791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.135793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.136248 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.136473 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.136698 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.136890 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":352,"spinlock_wait_cycles":3456,"thread_start_us":245,"threads_started":1}
I20260812 17:48:38.137786 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.124839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33386)
0812 17:48:38.125274 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.125314 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.125465 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.125825 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.125831 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.126314 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.126672 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.126685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.127785 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.127788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.128978 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.128993 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.129662 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.129664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.129882 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.130648 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.130672 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.132550 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.134732 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.134744 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.134756 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.135059 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.135385 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.135388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.135390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.135519 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.136030 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.136040 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.136260 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.136448 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.136820 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":21,"spinlock_wait_cycles":1792,"thread_start_us":115,"threads_started":1}
I20260812 17:48:38.139169 24587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.60@KRBTEST.COM for oryx/127.13.38.61@KRBTEST.COM
I20260812 17:48:38.147842 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.139752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:41603 (local address 127.0.0.1:33816)
0812 17:48:38.140166 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.140183 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.140269 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.140553 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.140580 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.140756 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.140854 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.140860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.141806 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.141811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.142358 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.142366 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.142837 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.142841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.142946 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.143484 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.143496 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.144796 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.146492 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.146495 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.146496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.146690 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.146937 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.146939 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.146940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.147012 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.147377 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.147379 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.147461 (+ 82us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.147609 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.147720 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334}
I20260812 17:48:38.147934 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.139908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33816 (local address 127.13.38.61:41603)
0812 17:48:38.140215 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.140222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.140238 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:48:38.140386 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.140393 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.140439 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.140537 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.140992 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.141692 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.142491 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.142727 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.142837 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.144932 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.144936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.144938 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.144963 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.146384 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.146776 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.146778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.146779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.146856 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.147112 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.147115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.147116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.147295 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.147372 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.147661 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.147768 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":114,"threads_started":1}
I20260812 17:48:38.148720 24587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.60@KRBTEST.COM for oryx/127.13.38.60@KRBTEST.COM
I20260812 17:48:38.158591 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.149304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:42247 (local address 127.0.0.1:37950)
0812 17:48:38.149485 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.149501 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.149585 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.150035 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.150038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.150236 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.150332 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.150339 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.151422 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.151426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.152367 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.152379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.153012 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.153015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.153088 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.153567 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.153578 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.154856 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.157033 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.157036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.157037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.157246 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.157595 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.157597 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.157599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.157685 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.158120 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.158122 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.158211 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.158322 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.158458 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 17:48:38.158712 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.149676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37950 (local address 127.13.38.60:42247)
0812 17:48:38.149800 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.149804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.149817 (+ 13us) server_negotiation.cc:407] Connection header received
0812 17:48:38.149852 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.149855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.149899 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.149987 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.150462 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.151220 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.152513 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.152901 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.152991 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.155051 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.155054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.155055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.155077 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.156857 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.157369 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.157372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.157373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.157471 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.157801 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.157804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.157805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.157998 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.158098 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.158405 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.158526 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 17:48:38.162235 24587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: Bootstrap starting.
I20260812 17:48:38.164708 24587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:38.165455 24587 log.cc:824] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:38.167665 24587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: No bootstrap required, opened a new log
I20260812 17:48:38.170786 24587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } }
I20260812 17:48:38.171058 24587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:38.171145 24587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41fe185daa47f696e387ee7b9f1bd5, State: Initialized, Role: FOLLOWER
I20260812 17:48:38.171663 24587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } }
I20260812 17:48:38.172214 24599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } } }
I20260812 17:48:38.172354 24599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:48:38.173013 24587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 17:48:38.178301 24617 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:48:38.178411 24617 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:48:38.182807 24342 master_runner.cc:428] Detected that this master 5b41fe185daa47f696e387ee7b9f1bd5 is joining an existing cluster
I20260812 17:48:38.182879 24342 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.38.60:42247
I20260812 17:48:38.191632 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 23547
I20260812 17:48:38.195117 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.186025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37966 (local address 127.13.38.60:42247)
0812 17:48:38.186137 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.186141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.186357 (+ 216us) server_negotiation.cc:407] Connection header received
0812 17:48:38.186485 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.186487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.186534 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.186581 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.187558 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.188279 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.189688 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.190036 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.190126 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.191780 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.191784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.191786 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.191809 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.193466 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.193902 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.193907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.193911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.193971 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.194213 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.194218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.194222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.194433 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.194496 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.194788 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.194910 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 17:48:38.195726 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.185696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:42247 (local address 127.0.0.1:37966)
0812 17:48:38.186288 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.186302 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.186400 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.186707 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.186709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.186958 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.187416 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.187424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.188413 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.188416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.189565 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.189578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.190340 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.190343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.190434 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.191055 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.191071 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.191642 (+ 571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.193593 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.193598 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.193602 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.193822 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.194055 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.194059 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.194063 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.194132 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.194561 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.194565 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.194651 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.195320 (+ 669us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.195464 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":322,"thread_start_us":102,"threads_started":1}
I20260812 17:48:38.226637 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.186433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33832 (local address 127.13.38.61:41603)
0812 17:48:38.186563 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.186568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.186844 (+ 276us) server_negotiation.cc:407] Connection header received
0812 17:48:38.187057 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.187060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.187120 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.187216 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.187840 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.188696 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.190023 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.190418 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.190501 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.222508 (+ 32007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.222513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.222517 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.222550 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.224570 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.225190 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.225194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.225196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.225285 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.225582 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.225585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.225588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.225834 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.225905 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.226290 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.226440 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 17:48:38.227492 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.186090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:41603 (local address 127.0.0.1:33832)
0812 17:48:38.186753 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.186769 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.186914 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.187263 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.187266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.187498 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.187678 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.187686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.188879 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.188889 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.189816 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.189827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.190670 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.190674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.190763 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.221711 (+ 30948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.221733 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.222280 (+ 547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.224740 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.224743 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.224745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.225063 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.225398 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.225401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.225403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.225486 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.225978 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.225981 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.226100 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.227121 (+ 1021us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.227296 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":1,"thread_start_us":73,"threads_started":1}
I20260812 17:48:38.230087 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.185696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33396 (local address 127.13.38.62:38917)
0812 17:48:38.185854 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.185858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.186131 (+ 273us) server_negotiation.cc:407] Connection header received
0812 17:48:38.186347 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.186353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.186410 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.186470 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.187304 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.188180 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.191281 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.191651 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.191741 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.225646 (+ 33905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.225653 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.225659 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.225699 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.227752 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.228371 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.228378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.228383 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.228491 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.228868 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.228874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.228878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.229217 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.229337 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.229805 (+ 468us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.229962 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:48:38.230386 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:45161
--local_ip_for_outbound_sockets=127.13.38.1
--tserver_master_addrs=127.13.38.62:38917
--webserver_port=36315
--webserver_interface=127.13.38.1
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab
--principal=oryx/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.38.62:38917,127.13.38.61:41603
--tserver_master_addrs=127.13.38.62:38917,127.13.38.61:41603,127.13.38.60:42247 with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:48:38.230654 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.185617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33396)
0812 17:48:38.186041 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.186058 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.186215 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.186625 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.186627 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.186958 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.187163 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.187174 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.188319 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.188325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.190137 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.190146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.222441 (+ 32295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.222448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.222625 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.224816 (+ 2191us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.224838 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.225426 (+ 588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.227921 (+ 2495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.227926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.227931 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.228232 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.228620 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.228626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.228630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.228733 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.229451 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.229456 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.229592 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.230272 (+ 680us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.230469 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":16,"thread_start_us":109,"threads_started":1}
I20260812 17:48:38.230726 24599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:38.230904 24599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } }
I20260812 17:48:38.232129 24599 leader_election.cc:288] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0104f86e900746688510a9947d247481 (127.13.38.62:38917), df1d07a897684999acc8fda17f1b348e (127.13.38.61:41603)
I20260812 17:48:38.233059 24409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df1d07a897684999acc8fda17f1b348e" is_pre_election: true
I20260812 17:48:38.233183 23464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0104f86e900746688510a9947d247481" is_pre_election: true
I20260812 17:48:38.233285 24409 raft_consensus.cc:1799] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 LEARNER]: Handling vote request from an unknown peer 5b41fe185daa47f696e387ee7b9f1bd5
I20260812 17:48:38.233433 24409 raft_consensus.cc:2396] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 5b41fe185daa47f696e387ee7b9f1bd5 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260812 17:48:38.234692 24506 leader_election.cc:302] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b41fe185daa47f696e387ee7b9f1bd5; no voters: 0104f86e900746688510a9947d247481, df1d07a897684999acc8fda17f1b348e
I20260812 17:48:38.235019 24599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:38.236558 24599 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556916, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.62@KRBTEST.COM for oryx/127.13.38.60@KRBTEST.COM
I20260812 17:48:38.245633 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.235474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37972 (local address 127.13.38.60:42247)
0812 17:48:38.235615 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.235620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.235634 (+ 14us) server_negotiation.cc:407] Connection header received
0812 17:48:38.235709 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.235712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.235766 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.235853 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.236367 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.237160 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.238345 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.238612 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.238714 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.241426 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.241430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.241434 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.241481 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.243415 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.244115 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.244118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.244120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.244212 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.244566 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.244569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.244571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.244812 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.244913 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.245299 (+ 386us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.245426 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 17:48:38.245671 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.235223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:42247 (local address 127.0.0.1:37972)
0812 17:48:38.235356 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.235372 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.235471 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.235895 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.235898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.236110 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.236225 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.236232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.237294 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.237298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.238204 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.238216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.238738 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.238742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.238843 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.239382 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.239396 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.241248 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.243683 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.243687 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.243689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.243990 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.244327 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.244333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.244337 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.244451 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.244929 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.244935 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.245090 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.245230 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.245427 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 17:48:38.247292 23451 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.38.60:42247: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5"
member_type: NON_VOTER
last_known_addr {
host: "127.13.38.60"
port: 42247
}
attrs {
promote: true
}
}
dest_uuid: "0104f86e900746688510a9947d247481"
cas_config_opid_index: 5
I20260812 17:48:38.248369 23451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } }
I20260812 17:48:38.250162 24336 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5b41fe185daa47f696e387ee7b9f1bd5 (127.13.38.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.251842 24336 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.38.60:42247
I20260812 17:48:38.252050 24336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.252147 24336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:38.252658 24342 master.cc:537] Master@127.13.38.60:42247 shutting down...
W20260812 17:48:38.255800 23434 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 -> Peer 5b41fe185daa47f696e387ee7b9f1bd5 (127.13.38.60:42247): Couldn't send request to peer 5b41fe185daa47f696e387ee7b9f1bd5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 17:48:38.255789 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.250101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.13.38.61:41603)
0812 17:48:38.250223 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.250228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.250385 (+ 157us) server_negotiation.cc:407] Connection header received
0812 17:48:38.250694 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.250697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.250767 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.250840 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:38.251325 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.252173 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.254105 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.254792 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.254873 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.255040 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.255504 (+ 464us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.255619 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 17:48:38.256569 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.249849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:41603 (local address 127.0.0.1:33848)
0812 17:48:38.250319 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.250334 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.250457 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.250992 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.250995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.251012 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:38.251101 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.251125 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.252420 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.252424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.253945 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.253958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.255143 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.255146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.255301 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.255386 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.256226 (+ 840us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.256308 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393}
I20260812 17:48:38.257376 24409 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 LEARNER]: Refusing update from remote peer 0104f86e900746688510a9947d247481: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 17:48:38.257885 24337 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 17:48:38.260171 24342 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:38.260469 24342 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:38.260557 24342 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: stopping tablet replica
I20260812 17:48:38.260201 24409 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5b41fe185daa47f696e387ee7b9f1bd5 (127.13.38.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.262737 24459 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0104f86e900746688510a9947d247481. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.262892 24459 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: This master's current role is: LEARNER
I20260812 17:48:38.263001 24459 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: NON_VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: true } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.263118 24459 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: This master's current role is: LEARNER
I20260812 17:48:38.264801 24337 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481: attempting to promote NON_VOTER df1d07a897684999acc8fda17f1b348e to VOTER
I20260812 17:48:38.265343 24337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } }
I20260812 17:48:38.267422 24409 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 LEARNER]: Refusing update from remote peer 0104f86e900746688510a9947d247481: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 17:48:38.269134 24336 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 17:48:38.270006 24342 master.cc:552] Master@127.13.38.60:42247 shutdown complete.
I20260812 17:48:38.270150 24342 master_runner.cc:305] Clearing existing system tablet
I20260812 17:48:38.270457 24342 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:38.270555 24342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:38.271134 24336 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, df1d07a897684999acc8fda17f1b348e (127.13.38.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.272001 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.266868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:42247 (local address 127.0.0.1:37978)
0812 17:48:38.271820 (+ 4952us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.271913 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:42247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4853}
I20260812 17:48:38.272763 24336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.272879 24336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:38.272991 24336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.273064 24336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:38.273532 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.272934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:42247 (local address 127.0.0.1:37990)
0812 17:48:38.273394 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.273449 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:42247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":108,"threads_started":1}
W20260812 17:48:38.273705 23434 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 -> Peer 5b41fe185daa47f696e387ee7b9f1bd5 (127.13.38.60:42247): Couldn't send request to peer 5b41fe185daa47f696e387ee7b9f1bd5. Status: Network error: Client connection negotiation failed: client connection to 127.13.38.60:42247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 17:48:38.274027 24409 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, df1d07a897684999acc8fda17f1b348e (127.13.38.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.274264 24342 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260812 17:48:38.275722 24631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:38.275723 24628 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.275868 24628 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:48:38.276079 24342 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:38.276201 24342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:38.276235 24342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/wal
uuid: "5b41fe185daa47f696e387ee7b9f1bd5"
format_stamp: "Formatted at 2026-08-12 17:48:38 on dist-test-slave-sb2z"
server_key: "5a68f52cda451873a26443adbb4d0f9e"
server_key_iv: "aac57aaebf33e4297eb4bf098023d565"
server_key_version: "encryptionkey@0"
I20260812 17:48:38.278386 24342 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:38.281219 24342 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 17:48:38.281335 24342 log.cc:1196] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 17:48:38.281621 24342 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: Deleting consensus metadata
I20260812 17:48:38.281960 24342 master_runner.cc:315] Copying system tablet from 127.13.38.62:38917
I20260812 17:48:38.283423 24342 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: tablet copy: Beginning tablet copy session from remote peer at address 127.13.38.62:38917
I20260812 17:48:38.297869 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.288052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33402)
0812 17:48:38.288479 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.288496 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.288638 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.289159 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.289162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.289405 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.289576 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.289584 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.290693 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.290697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.291677 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.291692 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.292345 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.292349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.292462 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.293141 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.293157 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.293646 (+ 489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.295850 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.295854 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.295856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.296130 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.296594 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.296597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.296599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.296772 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.297253 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.297257 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.297369 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.297486 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.297648 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":116,"threads_started":1}
I20260812 17:48:38.297917 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.288144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33402 (local address 127.13.38.62:38917)
0812 17:48:38.288295 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.288300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.288601 (+ 301us) server_negotiation.cc:407] Connection header received
0812 17:48:38.288735 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.288738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.288797 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.288888 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.289718 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.290554 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.291822 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.292205 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.292316 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.293815 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.293820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.293823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.293854 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.295718 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.296297 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.296304 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.296306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.296482 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.296873 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.296877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.296881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.297150 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.297252 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.297568 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.297706 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 17:48:38.298828 23483 tablet_copy_service.cc:140] P 0104f86e900746688510a9947d247481: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5b41fe185daa47f696e387ee7b9f1bd5 ({username='oryx', principal='oryx/127.13.38.60@KRBTEST.COM'} at 127.0.0.1:33402)
I20260812 17:48:38.298954 23483 tablet_copy_service.cc:161] P 0104f86e900746688510a9947d247481: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5b41fe185daa47f696e387ee7b9f1bd5 at {username='oryx', principal='oryx/127.13.38.60@KRBTEST.COM'} at 127.0.0.1:33402: session id = 5b41fe185daa47f696e387ee7b9f1bd5-00000000000000000000000000000000
I20260812 17:48:38.300122 23483 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 17:48:38.301636 24342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 17:48:38.304672 24342 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: tablet copy: Starting download of 0 data blocks...
I20260812 17:48:38.304983 24342 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: tablet copy: Starting download of 1 WAL segments...
I20260812 17:48:38.306761 24342 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 17:48:38.309118 23483 tablet_copy_service.cc:342] P 0104f86e900746688510a9947d247481: Request end of tablet copy session 5b41fe185daa47f696e387ee7b9f1bd5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.38.60@KRBTEST.COM'} at 127.0.0.1:33402
I20260812 17:48:38.309238 23483 tablet_copy_service.cc:433] P 0104f86e900746688510a9947d247481: ending tablet copy session 5b41fe185daa47f696e387ee7b9f1bd5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5b41fe185daa47f696e387ee7b9f1bd5
I20260812 17:48:38.317205 24342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:38.319096 24643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:38.319331 24644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:38.319710 24646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:38.319818 24342 server_base.cc:1034] running on GCE node
I20260812 17:48:38.320046 24342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:38.320377 24342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:38.321547 24342 hybrid_clock.cc:648] HybridClock initialized: now 1786556918321524 us; error 42 us; skew 500 ppm
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:38.323200 24342 init.cc:377] Logged in from keytab as oryx/127.13.38.60@KRBTEST.COM (short username oryx)
I20260812 17:48:38.323737 24342 webserver.cc:460] Webserver started at http://127.13.38.60:36391/ using document root <none> and password file <none>
I20260812 17:48:38.323940 24342 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:38.323989 24342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:38.325487 24342 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:38.326656 24657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:38.326930 24342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:38.327009 24342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/wal
uuid: "5b41fe185daa47f696e387ee7b9f1bd5"
format_stamp: "Formatted at 2026-08-12 17:48:38 on dist-test-slave-sb2z"
server_key: "5a68f52cda451873a26443adbb4d0f9e"
server_key_iv: "aac57aaebf33e4297eb4bf098023d565"
server_key_version: "encryptionkey@0"
I20260812 17:48:38.327091 24342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:38.328768 23964 heartbeater.cc:503] Master 127.13.38.62:38917 was elected leader, sending a full tablet report...
I20260812 17:48:38.364317 24342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:38.366703 24342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:38.372144 24342 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.60:42247
I20260812 17:48:38.372627 24745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.60:42247 every 8 connection(s)
I20260812 17:48:38.372866 24342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
I20260812 17:48:38.374953 24746 sys_catalog.cc:263] Verifying existing consensus state
I20260812 17:48:38.376130 24746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: Bootstrap starting.
I20260812 17:48:38.387295 24746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 17:48:38.387631 24746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: Bootstrap complete.
I20260812 17:48:38.388221 24746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } }
I20260812 17:48:38.388370 24746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b41fe185daa47f696e387ee7b9f1bd5, State: Initialized, Role: LEARNER
I20260812 17:48:38.388497 24746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } }
I20260812 17:48:38.388867 24750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.388943 24746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:38.389014 24750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: This master's current role is: LEARNER
I20260812 17:48:38.391821 24769 catalog_manager.cc:1259] Loaded cluster ID: dfa9bbf061bc403f8452108c10365386
I20260812 17:48:38.391945 24769 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: loading cluster ID for follower catalog manager: success
I20260812 17:48:38.393579 24769 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: acquiring CA information for follower catalog manager: success
I20260812 17:48:38.394658 24769 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 17:48:38.400396 24626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:38.400641 24626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:38.400723 24626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:38.404930 24626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:38.404986 24626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:38.405076 24626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:38.410710 24626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab
--principal=oryx/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:45161
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=36315
--tserver_master_addrs=127.13.38.62:38917,127.13.38.61:41603,127.13.38.60:42247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:38.412048 24626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:38.413185 24626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:38.423836 24776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:38.423887 24777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:38.423862 24626 server_base.cc:1034] running on GCE node
W20260812 17:48:38.423836 24779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:38.424587 24626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:38.425328 24626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:38.426535 24626 hybrid_clock.cc:648] HybridClock initialized: now 1786556918426515 us; error 35 us; skew 500 ppm
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:38.429637 24626 init.cc:377] Logged in from keytab as oryx/127.13.38.1@KRBTEST.COM (short username oryx)
I20260812 17:48:38.430553 24626 webserver.cc:460] Webserver started at http://127.13.38.1:36315/ using document root <none> and password file <none>
I20260812 17:48:38.431102 24626 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:38.431175 24626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:38.435519 24626 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:38.437817 24788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:38.439319 24626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:38.439429 24626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "e25783f400113d4fc8bc7e08de1de92d"
server_key_iv: "a0f1858f36c57f783bfa7ced76800d00"
server_key_version: "encryptionkey@0"
I20260812 17:48:38.440070 24626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:38.452059 24626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:38.455873 24626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:38.456063 24626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:38.456696 24626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:38.457571 24626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:38.457621 24626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:38.457662 24626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:38.457683 24626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:38.469255 24626 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:45161
I20260812 17:48:38.469297 24960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:45161 every 8 connection(s)
I20260812 17:48:38.470355 24626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:38.475847 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 24626
I20260812 17:48:38.476058 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 23767
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.61@KRBTEST.COM
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.60@KRBTEST.COM
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:38.490281 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.472499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:58253 (local address 127.13.38.62:38917)
0812 17:48:38.472648 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.472653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.473767 (+ 1114us) server_negotiation.cc:407] Connection header received
0812 17:48:38.475092 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.475095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.475158 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.475215 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.481261 (+ 6046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.482044 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.483250 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.483607 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.483689 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.486281 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.486286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.486289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.486330 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.488072 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.488554 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.488558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.488559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.488674 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.489014 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.489017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.489019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.489253 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.489322 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.489961 (+ 639us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.490099 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 17:48:38.490281 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.474615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:35839 (local address 127.13.38.61:41603)
0812 17:48:38.474745 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.474751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.474770 (+ 19us) server_negotiation.cc:407] Connection header received
0812 17:48:38.475064 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.475067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.475129 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.475216 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.476966 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.477683 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.479334 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.479727 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.479801 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.484586 (+ 4785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.484594 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.484599 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.484633 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.487115 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.488136 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.488141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.488143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.488241 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.488751 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.488757 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.488760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.489038 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.489124 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.489974 (+ 850us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.490099 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 17:48:38.490918 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.472658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:43793 (local address 127.13.38.60:42247)
0812 17:48:38.472948 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.472962 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.474035 (+ 1073us) server_negotiation.cc:407] Connection header received
0812 17:48:38.475179 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.475184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.475242 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.475322 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.478739 (+ 3417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.479804 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.481028 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.481557 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.481711 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.485206 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.485213 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.485219 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.485257 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.487342 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.488085 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.488091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.488096 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.488194 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.488677 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.488683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.488687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.488993 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.489104 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.489952 (+ 848us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.490135 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":93,"threads_started":1}
I20260812 17:48:38.492362 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.472981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:42247 (local address 127.13.38.1:43793)
0812 17:48:38.473946 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.473960 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.474844 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.475437 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.475440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.476076 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.478590 (+ 2514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.478598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.479956 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.479959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.480838 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.480851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.481884 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.481891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.482045 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.482735 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.482758 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.484982 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.487487 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.487493 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.487498 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.487948 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.488311 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.488317 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.488321 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.488548 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.489170 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.489181 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.489625 (+ 444us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.491289 (+ 1664us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.491442 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"mutex_wait_us":87,"spinlock_wait_cycles":10880,"thread_start_us":181,"threads_started":1}
I20260812 17:48:38.492667 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.472980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.1:58253)
0812 17:48:38.473601 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.473644 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.474844 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.475399 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.475407 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.476076 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.481101 (+ 5025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.481110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.482202 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.482205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.483099 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.483112 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.483866 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.483869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.483977 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.484497 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.484518 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.486097 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.488192 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.488197 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.488202 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.488437 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.488800 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.488804 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.488809 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.488888 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.489400 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.489403 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.489618 (+ 215us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.490570 (+ 952us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.491140 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":5098,"spinlock_wait_cycles":35968,"thread_start_us":683,"threads_started":1}
I20260812 17:48:38.494040 24961 heartbeater.cc:347] Connected to a master server at 127.13.38.60:42247
I20260812 17:48:38.494151 24962 heartbeater.cc:347] Connected to a master server at 127.13.38.62:38917
I20260812 17:48:38.494329 24961 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:38.494328 24962 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:38.494969 24961 heartbeater.cc:511] Master 127.13.38.60:42247 requested a full tablet report, sending...
I20260812 17:48:38.494972 24962 heartbeater.cc:511] Master 127.13.38.62:38917 requested a full tablet report, sending...
I20260812 17:48:38.496330 23451 ts_manager.cc:194] Re-registered known tserver with Master: b8e8d5ffa8c347f2a3ad8bc1e448d811 (127.13.38.1:45161)
I20260812 17:48:38.496634 24687 ts_manager.cc:194] Registered new tserver with Master: b8e8d5ffa8c347f2a3ad8bc1e448d811 (127.13.38.1:45161)
I20260812 17:48:38.492374 24974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.474260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:41603 (local address 127.13.38.1:35839)
0812 17:48:38.474512 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.474527 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.474852 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.475397 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.475404 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.476200 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.476796 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.476814 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.477831 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.477837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.479191 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.479201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.480013 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.480017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.480200 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.480912 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.480938 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.484050 (+ 3112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.487293 (+ 3243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.487301 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.487324 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.488003 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.488368 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.488373 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.488376 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.488552 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.489411 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.489414 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.489618 (+ 204us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.490570 (+ 952us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.491086 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":12,"spinlock_wait_cycles":12416,"thread_start_us":89,"threads_started":1}
I20260812 17:48:38.497124 23451 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:58253
I20260812 17:48:38.497534 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:35863
--local_ip_for_outbound_sockets=127.13.38.2
--tserver_master_addrs=127.13.38.62:38917
--webserver_port=44881
--webserver_interface=127.13.38.2
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab
--principal=oryx/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.38.62:38917,127.13.38.61:41603
--tserver_master_addrs=127.13.38.62:38917,127.13.38.61:41603,127.13.38.60:42247 with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:48:38.498041 24963 heartbeater.cc:347] Connected to a master server at 127.13.38.61:41603
I20260812 17:48:38.498183 24963 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:38.498558 24963 heartbeater.cc:511] Master 127.13.38.61:41603 requested a full tablet report, sending...
I20260812 17:48:38.499811 24397 ts_manager.cc:194] Registered new tserver with Master: b8e8d5ffa8c347f2a3ad8bc1e448d811 (127.13.38.1:45161)
I20260812 17:48:38.555266 24170 heartbeater.cc:503] Master 127.13.38.62:38917 was elected leader, sending a full tablet report...
W20260812 17:48:38.619285 24976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:38.619644 24976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:38.619728 24976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:38.623893 24976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:38.623960 24976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:38.624048 24976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:38.629711 24976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab
--principal=oryx/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:35863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=44881
--tserver_master_addrs=127.13.38.62:38917,127.13.38.61:41603,127.13.38.60:42247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:38.630766 24976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:38.631932 24976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:38.642364 24995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:38.642391 24990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:38.642576 24976 server_base.cc:1034] running on GCE node
W20260812 17:48:38.642372 24986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:38.643165 24976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:38.643890 24976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:38.645059 24976 hybrid_clock.cc:648] HybridClock initialized: now 1786556918645040 us; error 55 us; skew 500 ppm
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:38.648373 24976 init.cc:377] Logged in from keytab as oryx/127.13.38.2@KRBTEST.COM (short username oryx)
I20260812 17:48:38.649379 24976 webserver.cc:460] Webserver started at http://127.13.38.2:44881/ using document root <none> and password file <none>
I20260812 17:48:38.650010 24976 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:38.650125 24976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:38.654347 24976 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:38.656839 25003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:38.658248 24976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:38.658398 24976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "f15015c53e544edeb79ea507c2cd2b18"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "1235e64cd91265e2bcb4879a09eb7027"
server_key_iv: "ededee90916c20356226321063158238"
server_key_version: "encryptionkey@0"
I20260812 17:48:38.658983 24976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:38.679679 24976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:38.683848 24976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:38.684038 24976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:38.684748 24976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:38.685673 24976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:38.685724 24976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:38.685767 24976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:38.685791 24976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:38.695451 24976 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:35863
I20260812 17:48:38.695513 25176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:35863 every 8 connection(s)
I20260812 17:48:38.696626 24976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:38.705989 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 24976
I20260812 17:48:38.706172 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 23975
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.61@KRBTEST.COM
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.60@KRBTEST.COM
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:38.713862 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.700411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:34177 (local address 127.13.38.61:41603)
0812 17:48:38.700573 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.700578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.700667 (+ 89us) server_negotiation.cc:407] Connection header received
0812 17:48:38.701113 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.701117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.701172 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.701228 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.702561 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.703368 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.704789 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.705128 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.705207 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.709045 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.709051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.709055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.709084 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.710926 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.711683 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.711687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.711689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.711788 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.712273 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.712277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.712278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.712591 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.712676 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.713511 (+ 835us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.713666 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 17:48:38.714890 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.698828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:48529 (local address 127.13.38.62:38917)
0812 17:48:38.698966 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.698971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.699827 (+ 856us) server_negotiation.cc:407] Connection header received
0812 17:48:38.701113 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.701117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.701172 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.701228 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.702561 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.703380 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.704789 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.705140 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.705210 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.711108 (+ 5898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.711113 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.711115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.711150 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.712957 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.713451 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.713455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.713457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.713548 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.713854 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.713858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.713859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.714102 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.714185 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.714610 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.714730 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 17:48:38.715683 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.699135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.2:48529)
0812 17:48:38.699662 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.699719 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.700906 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.701441 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.701448 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.701935 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.702406 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.702415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.703546 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.703549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.704648 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.704655 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.705371 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.705374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.705536 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.708944 (+ 3408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.708963 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.710900 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.713091 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.713095 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.713097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.713331 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.713666 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.713669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.713670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.713752 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.714321 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.714323 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.714412 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.715117 (+ 705us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.715303 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":284,"spinlock_wait_cycles":45824,"thread_start_us":150,"threads_started":1}
I20260812 17:48:38.715970 25187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.700254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:41603 (local address 127.13.38.2:34177)
0812 17:48:38.700597 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.700609 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.700906 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.701441 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.701448 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.701929 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.702409 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.702422 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.703546 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.703549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.704644 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.704655 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.705371 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.705374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.705535 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.706179 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.706195 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.708439 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.711063 (+ 2624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.711071 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.711091 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.711516 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.711921 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.711924 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.711929 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.712147 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.712775 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.712782 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.713179 (+ 397us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.714064 (+ 885us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.714486 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":237,"spinlock_wait_cycles":4480,"thread_start_us":115,"threads_started":1}
I20260812 17:48:38.717219 25181 heartbeater.cc:347] Connected to a master server at 127.13.38.62:38917
I20260812 17:48:38.717316 25182 heartbeater.cc:347] Connected to a master server at 127.13.38.61:41603
I20260812 17:48:38.717483 25181 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:38.717483 25182 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:38.718202 25181 heartbeater.cc:511] Master 127.13.38.62:38917 requested a full tablet report, sending...
I20260812 17:48:38.718202 25182 heartbeater.cc:511] Master 127.13.38.61:41603 requested a full tablet report, sending...
I20260812 17:48:38.719431 23451 ts_manager.cc:194] Re-registered known tserver with Master: f15015c53e544edeb79ea507c2cd2b18 (127.13.38.2:35863)
I20260812 17:48:38.719556 24397 ts_manager.cc:194] Registered new tserver with Master: f15015c53e544edeb79ea507c2cd2b18 (127.13.38.2:35863)
I20260812 17:48:38.720194 23451 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:48529
I20260812 17:48:38.733789 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.699135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:42247 (local address 127.13.38.2:56241)
0812 17:48:38.699662 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.699719 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.700855 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.701439 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.701446 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.701873 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.702389 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.702402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.703251 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.703255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.704225 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.704235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.704794 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.704798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.704982 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.705644 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.705727 (+ 83us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.708859 (+ 3132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.731894 (+ 23035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.731899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.731900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.732163 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.732569 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.732572 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.732573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.732670 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.733235 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.733238 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.733356 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.733453 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.733621 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":33,"spinlock_wait_cycles":126592,"thread_start_us":157,"threads_started":1}
I20260812 17:48:38.734001 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.698624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:56241 (local address 127.13.38.60:42247)
0812 17:48:38.698800 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.698805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.699829 (+ 1024us) server_negotiation.cc:407] Connection header received
0812 17:48:38.701106 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.701109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.701177 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.701259 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.702552 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.703097 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.704364 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.704629 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.704751 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.729580 (+ 24829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.729584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.729587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.729632 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.731728 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.732315 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.732322 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.732327 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.732442 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.732784 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.732790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.732794 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.733098 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.733211 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.733553 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.733744 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 17:48:38.737242 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:32929
--local_ip_for_outbound_sockets=127.13.38.3
--tserver_master_addrs=127.13.38.62:38917
--webserver_port=39913
--webserver_interface=127.13.38.3
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab
--principal=oryx/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.38.62:38917,127.13.38.61:41603
--tserver_master_addrs=127.13.38.62:38917,127.13.38.61:41603,127.13.38.60:42247 with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:48:38.737526 25177 heartbeater.cc:347] Connected to a master server at 127.13.38.60:42247
I20260812 17:48:38.737721 25177 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:38.738092 25177 heartbeater.cc:511] Master 127.13.38.60:42247 requested a full tablet report, sending...
I20260812 17:48:38.739138 24687 ts_manager.cc:194] Registered new tserver with Master: f15015c53e544edeb79ea507c2cd2b18 (127.13.38.2:35863)
I20260812 17:48:38.825315 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.820916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:42247 (local address 127.0.0.1:38002)
0812 17:48:38.821244 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.821257 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.821369 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.821721 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.821724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.821745 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:38.821845 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.821850 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.822938 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.822942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.823868 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.823881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.824897 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.824900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.824998 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.825025 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.825101 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.825177 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":67,"threads_started":1}
I20260812 17:48:38.825327 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.821076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38002 (local address 127.13.38.60:42247)
0812 17:48:38.821234 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.821240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.821332 (+ 92us) server_negotiation.cc:407] Connection header received
0812 17:48:38.821481 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.821484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.821554 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.821687 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:38.821971 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.822794 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.824001 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.824735 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.824856 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.825030 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.825119 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.825193 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:48:38.826195 24705 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 17:48:38.826694 24750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0104f86e900746688510a9947d247481. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.826848 24750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: This master's current role is: LEARNER
W20260812 17:48:38.868637 25189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:38.868906 25189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:38.868952 25189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:38.873027 25189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:38.873095 25189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:38.873178 25189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:38.878542 25189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab
--principal=oryx/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:32929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=39913
--tserver_master_addrs=127.13.38.62:38917,127.13.38.61:41603,127.13.38.60:42247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:38.879783 25189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:38.880859 25189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:38.892119 25205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:38.892113 25200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:38.892167 25202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:38.892616 25189 server_base.cc:1034] running on GCE node
I20260812 17:48:38.893105 25189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:38.893806 25189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:38.894969 25189 hybrid_clock.cc:648] HybridClock initialized: now 1786556918894932 us; error 55 us; skew 500 ppm
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:38.898131 25189 init.cc:377] Logged in from keytab as oryx/127.13.38.3@KRBTEST.COM (short username oryx)
I20260812 17:48:38.899073 25189 webserver.cc:460] Webserver started at http://127.13.38.3:39913/ using document root <none> and password file <none>
I20260812 17:48:38.899761 25189 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:38.899879 25189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:38.904119 25189 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:48:38.906551 25214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:38.907945 25189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:38.908099 25189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "1250825c07de44a2af1c6a4a1487de2c"
format_stamp: "Formatted at 2026-08-12 17:48:37 on dist-test-slave-sb2z"
server_key: "b396f568d44d53449f18e077356ac8b0"
server_key_iv: "e5f5fc5a8f317506911ee204328ae93b"
server_key_version: "encryptionkey@0"
I20260812 17:48:38.908699 25189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:38.920677 25189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:38.923951 25189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:38.924162 25189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:38.924929 25189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:38.925890 25189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:38.925962 25189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:38.926038 25189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:38.926080 25189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:38.935778 25189 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:32929
I20260812 17:48:38.935830 25378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:32929 every 8 connection(s)
I20260812 17:48:38.936920 25189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:38.945926 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 25189
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.60@KRBTEST.COM
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.61@KRBTEST.COM
Aug 12 17:48:38 dist-test-slave-sb2z krb5kdc[23380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556918, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:38.956754 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.938975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:42951 (local address 127.13.38.60:42247)
0812 17:48:38.939118 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.939124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.940512 (+ 1388us) server_negotiation.cc:407] Connection header received
0812 17:48:38.941893 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.941896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.941960 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.942013 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.943696 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.944554 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.946262 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.946595 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.946665 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.951887 (+ 5222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.951894 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.951900 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.951940 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.954710 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.955204 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.955207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.955209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.955294 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.955629 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.955634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.955639 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.955854 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.955950 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.956448 (+ 498us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.956584 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 17:48:38.956754 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.938975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:55251 (local address 127.13.38.62:38917)
0812 17:48:38.939118 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.939124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.940512 (+ 1388us) server_negotiation.cc:407] Connection header received
0812 17:48:38.941892 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.941896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.941957 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.942013 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.943749 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.944465 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.946181 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.946529 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.946618 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.952026 (+ 5408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.952030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.952033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.952064 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.954139 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.954727 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.954731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.954734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.954828 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.955170 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.955175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.955180 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.955451 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.955593 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.956471 (+ 878us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.956582 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 17:48:38.957810 25386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.939618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.3:55251)
0812 17:48:38.940315 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.940384 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.941630 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.942250 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.942258 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.942772 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.943509 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.943518 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.944623 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.944628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.946033 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.946043 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.946809 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.946813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.947143 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.948021 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.948046 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.951865 (+ 3819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.954290 (+ 2425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.954294 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.954297 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.954602 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.954935 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.954938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.954940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.955031 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.955688 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.955691 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.955913 (+ 222us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.956141 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.957432 (+ 1291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":1331,"spinlock_wait_cycles":102528,"thread_start_us":163,"threads_started":1}
I20260812 17:48:38.958117 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.940910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:41603 (local address 127.13.38.3:42011)
0812 17:48:38.941209 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.941232 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.941636 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.942248 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.942256 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.942764 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.943500 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.943517 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.944715 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.944718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.945888 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.945901 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.946805 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.946808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.947621 (+ 813us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.948209 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.948224 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.951282 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.953745 (+ 2463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.953755 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.953779 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.954228 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.954614 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.954616 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.954618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.954829 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.955481 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.955491 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.955917 (+ 426us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.956143 (+ 226us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.956610 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":745,"spinlock_wait_cycles":43520,"thread_start_us":126,"threads_started":1}
I20260812 17:48:38.958604 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.939618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:42247 (local address 127.13.38.3:42951)
0812 17:48:38.940315 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.940384 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.941630 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.942251 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.942258 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.942772 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:38.943509 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.943518 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.944648 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.944651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.946031 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.946043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.946814 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.946817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.947142 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.948377 (+ 1235us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:38.948388 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:38.951705 (+ 3317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:38.954829 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.954833 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.954835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.955090 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.955403 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:38.955406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:38.955408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:38.955504 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:38.956007 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:38.956010 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:38.956088 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.956192 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.958405 (+ 2213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":2580,"spinlock_wait_cycles":30848,"thread_start_us":152,"threads_started":1}
I20260812 17:48:38.959262 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.941045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:42011 (local address 127.13.38.61:41603)
0812 17:48:38.941178 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.941183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.941315 (+ 132us) server_negotiation.cc:407] Connection header received
0812 17:48:38.941858 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.941862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.941917 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.941999 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.943729 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.944555 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.946130 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.946484 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.946576 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.951467 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.951471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.951474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.951504 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.953593 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.954359 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.954362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.954363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.954489 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.954963 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.954988 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.954991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.955285 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.955387 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.958943 (+ 3556us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.959109 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:48:38.959324 25382 heartbeater.cc:347] Connected to a master server at 127.13.38.62:38917
I20260812 17:48:38.959604 25382 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:38.960296 25382 heartbeater.cc:511] Master 127.13.38.62:38917 requested a full tablet report, sending...
I20260812 17:48:38.960474 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.949415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33410 (local address 127.13.38.62:38917)
0812 17:48:38.950056 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.950063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.950078 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:48:38.950119 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.950122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.950170 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.950263 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.951135 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.951911 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.953302 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.953714 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.953809 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.955649 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.955652 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.955655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.955679 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.957420 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.958393 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.958397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.958398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.958499 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.959089 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.959093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.959095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.959593 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.959793 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.960154 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.960273 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":91,"threads_started":1}
I20260812 17:48:38.960927 25379 heartbeater.cc:347] Connected to a master server at 127.13.38.60:42247
I20260812 17:48:38.961035 25379 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:38.961246 25379 heartbeater.cc:511] Master 127.13.38.60:42247 requested a full tablet report, sending...
I20260812 17:48:38.962167 24687 ts_manager.cc:194] Registered new tserver with Master: 1250825c07de44a2af1c6a4a1487de2c (127.13.38.3:32929)
I20260812 17:48:38.963763 25383 heartbeater.cc:347] Connected to a master server at 127.13.38.61:41603
I20260812 17:48:38.963866 25383 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:38.964071 25383 heartbeater.cc:511] Master 127.13.38.61:41603 requested a full tablet report, sending...
I20260812 17:48:38.964625 23451 ts_manager.cc:194] Re-registered known tserver with Master: 1250825c07de44a2af1c6a4a1487de2c (127.13.38.3:32929)
I20260812 17:48:38.964989 24397 ts_manager.cc:194] Registered new tserver with Master: 1250825c07de44a2af1c6a4a1487de2c (127.13.38.3:32929)
I20260812 17:48:38.965307 23451 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:55251
I20260812 17:48:38.965807 23452 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 17:48:38.970309 25397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.949955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33850 (local address 127.13.38.61:41603)
0812 17:48:38.950223 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.950228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.959311 (+ 9083us) server_negotiation.cc:407] Connection header received
0812 17:48:38.959554 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.959557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.959609 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.959679 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.961016 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.961715 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.962875 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.963230 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.963332 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.966147 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.966151 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.966154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.966180 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.968013 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.968499 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.968506 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.968508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.968597 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.969079 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.969085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.969089 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.969442 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.969581 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.970008 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.970139 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260812 17:48:38.970412 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.952666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.13.38.60:42247)
0812 17:48:38.956940 (+ 4274us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.956944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.960456 (+ 3512us) server_negotiation.cc:407] Connection header received
0812 17:48:38.960646 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.960649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.960693 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.960739 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:38.961688 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.962262 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.963167 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.963597 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.963704 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.966595 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:38.966599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:38.966602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:38.966630 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:38.968308 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.968786 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.968790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.968792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.968931 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:38.969263 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:38.969267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:38.969268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:38.969556 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:38.969686 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.969992 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.970113 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4205,"thread_start_us":82,"threads_started":1}
W20260812 17:48:38.970618 23452 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:38.973711 25193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: NON_VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: true } } }
I20260812 17:48:38.973847 25193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:38.977612 25193 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481: attempting to promote NON_VOTER 5b41fe185daa47f696e387ee7b9f1bd5 to VOTER
I20260812 17:48:38.977946 25193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } }
I20260812 17:48:38.980895 24701 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 LEARNER]: Refusing update from remote peer 0104f86e900746688510a9947d247481: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 17:48:38.981415 25405 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 17:48:38.982429 24409 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 FOLLOWER]: Refusing update from remote peer 0104f86e900746688510a9947d247481: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 17:48:38.983898 25405 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 17:48:38.983839 25404 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5b41fe185daa47f696e387ee7b9f1bd5 (127.13.38.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } } }
I20260812 17:48:38.984612 24705 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5b41fe185daa47f696e387ee7b9f1bd5 (127.13.38.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } } }
I20260812 17:48:38.985411 24409 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5b41fe185daa47f696e387ee7b9f1bd5 (127.13.38.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } } }
I20260812 17:48:38.985759 25404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } } }
I20260812 17:48:38.985961 25404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:38.986104 25404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } } }
I20260812 17:48:38.986169 25404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:38.986982 24750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } } }
I20260812 17:48:38.987154 24750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:48:38.991741 25414 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } } }
I20260812 17:48:38.991901 25414 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:48:38.995059 25418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.988952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45161 (local address 127.0.0.1:59278)
0812 17:48:38.989873 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.989898 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.989992 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.990599 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.990602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.990622 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:38.990738 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.990744 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.992177 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.992180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.993327 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.993338 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.994615 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.994619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.994714 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.994742 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.994828 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.994907 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":3,"thread_start_us":119,"threads_started":1}
I20260812 17:48:38.995105 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.989142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59278 (local address 127.13.38.1:45161)
0812 17:48:38.989510 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.989531 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.990094 (+ 563us) server_negotiation.cc:407] Connection header received
0812 17:48:38.990191 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.990199 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.990458 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.990642 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:38.990887 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.992048 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.993469 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.994486 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.994597 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.994758 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.994862 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.994969 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":144,"threads_started":1}
I20260812 17:48:38.997001 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.988225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:35863 (local address 127.0.0.1:52756)
0812 17:48:38.988582 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.988599 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.988709 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.991843 (+ 3134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.991846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.991863 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:38.991954 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.991960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.994239 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.994242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.995304 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.995316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.996605 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.996610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.996695 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.996717 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.996767 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.996876 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271}
I20260812 17:48:38.997134 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.989611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:32929 (local address 127.0.0.1:47452)
0812 17:48:38.992337 (+ 2726us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:38.992353 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:38.992460 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:38.992888 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:38.992892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:38.992910 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:38.993001 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.993008 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.994158 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.994163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.995173 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:38.995185 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.996652 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.996656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:38.996745 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.996771 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 17:48:38.996857 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 17:48:38.996938 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2659,"thread_start_us":114,"threads_started":1}
I20260812 17:48:38.997139 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.989851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47452 (local address 127.13.38.3:32929)
0812 17:48:38.992173 (+ 2322us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.992184 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.992448 (+ 264us) server_negotiation.cc:407] Connection header received
0812 17:48:38.992588 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.992593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.992754 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.992914 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:38.993149 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.994030 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.995315 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.996534 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.996639 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.996799 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.996887 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.996986 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2215,"thread_start_us":198,"threads_started":1}
I20260812 17:48:38.997834 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:38.988944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52756 (local address 127.13.38.2:35863)
0812 17:48:38.991306 (+ 2362us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:38.991318 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:38.991343 (+ 25us) server_negotiation.cc:407] Connection header received
0812 17:48:38.991435 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:38.991441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:38.991679 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:38.992207 (+ 528us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:38.992355 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.994096 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.995674 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:38.996443 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:38.997389 (+ 946us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:38.997555 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:38.997607 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 17:48:38.997694 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2251,"thread_start_us":115,"threads_started":1}
I20260812 17:48:38.998654 24862 tablet_service.cc:1467] Processing CreateTablet for tablet 29dafadff54846aea52a77f3eadbc289 (DEFAULT_TABLE table=test-table [id=91c8055e046c408fbdb8d8faf40417c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:38.999845 24862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29dafadff54846aea52a77f3eadbc289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:39.001127 25082 tablet_service.cc:1467] Processing CreateTablet for tablet 29dafadff54846aea52a77f3eadbc289 (DEFAULT_TABLE table=test-table [id=91c8055e046c408fbdb8d8faf40417c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:39.001703 25279 tablet_service.cc:1467] Processing CreateTablet for tablet 29dafadff54846aea52a77f3eadbc289 (DEFAULT_TABLE table=test-table [id=91c8055e046c408fbdb8d8faf40417c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:39.002305 25082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29dafadff54846aea52a77f3eadbc289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:39.002746 25279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29dafadff54846aea52a77f3eadbc289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:39.007144 25420 tablet_bootstrap.cc:492] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811: Bootstrap starting.
I20260812 17:48:39.010190 25420 tablet_bootstrap.cc:654] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:39.011060 25420 log.cc:824] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:39.011453 25421 tablet_bootstrap.cc:492] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c: Bootstrap starting.
I20260812 17:48:39.012166 25422 tablet_bootstrap.cc:492] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18: Bootstrap starting.
I20260812 17:48:39.014045 25420 tablet_bootstrap.cc:492] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811: No bootstrap required, opened a new log
I20260812 17:48:39.014169 25421 tablet_bootstrap.cc:654] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:39.014290 25420 ts_tablet_manager.cc:1397] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 17:48:39.014781 25422 tablet_bootstrap.cc:654] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:39.015206 25421 log.cc:824] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:39.015873 25422 log.cc:824] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:39.017722 25421 tablet_bootstrap.cc:492] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c: No bootstrap required, opened a new log
I20260812 17:48:39.017742 25422 tablet_bootstrap.cc:492] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18: No bootstrap required, opened a new log
I20260812 17:48:39.017907 25422 ts_tablet_manager.cc:1397] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260812 17:48:39.017926 25421 ts_tablet_manager.cc:1397] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 17:48:39.018693 25420 raft_consensus.cc:348] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:39.019011 25420 raft_consensus.cc:374] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:39.019068 25420 raft_consensus.cc:729] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e8d5ffa8c347f2a3ad8bc1e448d811, State: Initialized, Role: FOLLOWER
I20260812 17:48:39.019608 25420 consensus_queue.cc:260] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:39.021109 25420 ts_tablet_manager.cc:1428] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260812 17:48:39.021445 24962 heartbeater.cc:503] Master 127.13.38.62:38917 was elected leader, sending a full tablet report...
I20260812 17:48:39.022641 25421 raft_consensus.cc:348] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:39.022929 25421 raft_consensus.cc:374] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:39.023002 25421 raft_consensus.cc:729] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1250825c07de44a2af1c6a4a1487de2c, State: Initialized, Role: FOLLOWER
I20260812 17:48:39.022234 25422 raft_consensus.cc:348] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:39.023453 25422 raft_consensus.cc:374] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:39.023586 25422 raft_consensus.cc:729] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f15015c53e544edeb79ea507c2cd2b18, State: Initialized, Role: FOLLOWER
I20260812 17:48:39.023550 25421 consensus_queue.cc:260] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:39.024220 25422 consensus_queue.cc:260] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:39.024662 25421 ts_tablet_manager.cc:1428] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260812 17:48:39.025334 25422 ts_tablet_manager.cc:1428] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260812 17:48:39.025362 25382 heartbeater.cc:503] Master 127.13.38.62:38917 was elected leader, sending a full tablet report...
I20260812 17:48:39.026624 25181 heartbeater.cc:503] Master 127.13.38.62:38917 was elected leader, sending a full tablet report...
I20260812 17:48:39.068315 25428 raft_consensus.cc:482] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:39.068598 25428 raft_consensus.cc:504] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:39.069865 25428 leader_election.cc:288] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f15015c53e544edeb79ea507c2cd2b18 (127.13.38.2:35863), b8e8d5ffa8c347f2a3ad8bc1e448d811 (127.13.38.1:45161)
I20260812 17:48:39.073773 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:39.070202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:33999 (local address 127.13.38.2:35863)
0812 17:48:39.070341 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:39.070347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:39.070437 (+ 90us) server_negotiation.cc:407] Connection header received
0812 17:48:39.070589 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:39.070596 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:39.070651 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:39.070711 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:39.071040 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.071658 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.072787 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.073175 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.073257 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:39.073350 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:39.073559 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 17:48:39.073636 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 17:48:39.073791 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:39.070140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45161 (local address 127.13.38.3:38887)
0812 17:48:39.070341 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:39.070356 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:39.070487 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:39.070823 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:39.070826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:39.070842 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:39.070943 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:39.070948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.071751 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.071756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:39.072659 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:39.072668 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.073350 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.073355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:39.073462 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:39.073482 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 17:48:39.073581 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 17:48:39.073647 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":36,"spinlock_wait_cycles":4864}
I20260812 17:48:39.073799 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:39.070280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:38887 (local address 127.13.38.1:45161)
0812 17:48:39.070412 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:39.070416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:39.070434 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:48:39.070639 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:39.070642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:39.070709 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:39.070794 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:39.071088 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.071641 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.072815 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.073196 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.073329 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:39.073395 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:39.073615 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 17:48:39.073695 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 17:48:39.073798 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:39.070110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:35863 (local address 127.13.38.3:33999)
0812 17:48:39.070342 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:39.070356 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:39.070470 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:39.070752 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:39.070757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:39.070779 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:39.070921 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:39.070927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.071786 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.071789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:39.072653 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:39.072663 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.073303 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.073307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:39.073417 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:39.073442 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 17:48:39.073511 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 17:48:39.073606 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":55}
I20260812 17:48:39.074476 24896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29dafadff54846aea52a77f3eadbc289" candidate_uuid: "1250825c07de44a2af1c6a4a1487de2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" is_pre_election: true
I20260812 17:48:39.074486 25108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29dafadff54846aea52a77f3eadbc289" candidate_uuid: "1250825c07de44a2af1c6a4a1487de2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f15015c53e544edeb79ea507c2cd2b18" is_pre_election: true
I20260812 17:48:39.074800 24896 raft_consensus.cc:2454] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1250825c07de44a2af1c6a4a1487de2c in term 0.
I20260812 17:48:39.074807 25108 raft_consensus.cc:2454] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1250825c07de44a2af1c6a4a1487de2c in term 0.
I20260812 17:48:39.075301 25220 leader_election.cc:302] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1250825c07de44a2af1c6a4a1487de2c, b8e8d5ffa8c347f2a3ad8bc1e448d811; no voters:
I20260812 17:48:39.075656 25428 raft_consensus.cc:2781] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:39.075737 25428 raft_consensus.cc:482] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:39.075768 25428 raft_consensus.cc:3037] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:39.076853 25428 raft_consensus.cc:504] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:39.077225 25428 leader_election.cc:288] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [CANDIDATE]: Term 1 election: Requested vote from peers f15015c53e544edeb79ea507c2cd2b18 (127.13.38.2:35863), b8e8d5ffa8c347f2a3ad8bc1e448d811 (127.13.38.1:45161)
I20260812 17:48:39.077590 24896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29dafadff54846aea52a77f3eadbc289" candidate_uuid: "1250825c07de44a2af1c6a4a1487de2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811"
I20260812 17:48:39.077590 25108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29dafadff54846aea52a77f3eadbc289" candidate_uuid: "1250825c07de44a2af1c6a4a1487de2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f15015c53e544edeb79ea507c2cd2b18"
I20260812 17:48:39.077713 25108 raft_consensus.cc:3037] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:39.077713 24896 raft_consensus.cc:3037] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:39.079030 24896 raft_consensus.cc:2454] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1250825c07de44a2af1c6a4a1487de2c in term 1.
I20260812 17:48:39.079066 25108 raft_consensus.cc:2454] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1250825c07de44a2af1c6a4a1487de2c in term 1.
I20260812 17:48:39.079398 25218 leader_election.cc:302] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1250825c07de44a2af1c6a4a1487de2c, b8e8d5ffa8c347f2a3ad8bc1e448d811, f15015c53e544edeb79ea507c2cd2b18; no voters:
I20260812 17:48:39.079610 25428 raft_consensus.cc:2781] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:39.080018 25428 raft_consensus.cc:686] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 1 LEADER]: Becoming Leader. State: Replica: 1250825c07de44a2af1c6a4a1487de2c, State: Running, Role: LEADER
I20260812 17:48:39.080435 25428 consensus_queue.cc:237] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:39.084474 23451 catalog_manager.cc:5495] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1250825c07de44a2af1c6a4a1487de2c (127.13.38.3). New cstate: current_term: 1 leader_uuid: "1250825c07de44a2af1c6a4a1487de2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } health_report { overall_health: HEALTHY } } }
I20260812 17:48:39.086398 25412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0104f86e900746688510a9947d247481" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } } }
I20260812 17:48:39.086537 25412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:39.106477 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:39.102656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47460 (local address 127.13.38.3:32929)
0812 17:48:39.102807 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:39.102811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:39.102826 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:48:39.102998 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:39.103000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:39.103064 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:39.103189 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:39.103441 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.104018 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.104925 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.105156 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.105266 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:39.105593 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:39.106056 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:39.106210 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:39.106252 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 17:48:39.106319 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 17:48:39.113174 25108 raft_consensus.cc:1261] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18 [term 1 FOLLOWER]: Refusing update from remote peer 1250825c07de44a2af1c6a4a1487de2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:39.113214 24896 raft_consensus.cc:1261] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 1 FOLLOWER]: Refusing update from remote peer 1250825c07de44a2af1c6a4a1487de2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:39.114013 25428 consensus_queue.cc:1035] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:39.114254 25431 consensus_queue.cc:1035] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:39.125259 25442 mvcc.cc:205] Tried to move back new op lower bound from 7317737140678979584 to 7317737140556328960. Current Snapshot: MvccSnapshot[applied={T|T < 7317737140678979584}]
I20260812 17:48:39.129328 25441 mvcc.cc:205] Tried to move back new op lower bound from 7317737140678979584 to 7317737140556328960. Current Snapshot: MvccSnapshot[applied={T|T < 7317737140678979584}]
I20260812 17:48:39.135789 23451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:39.136037 23451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:39.140758 23451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0104f86e900746688510a9947d247481: Sending DeleteTablet for 3 replicas of tablet 29dafadff54846aea52a77f3eadbc289
I20260812 17:48:39.141937 24862 tablet_service.cc:1514] Processing DeleteTablet for tablet 29dafadff54846aea52a77f3eadbc289 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:39 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:59278
I20260812 17:48:39.142266 25082 tablet_service.cc:1514] Processing DeleteTablet for tablet 29dafadff54846aea52a77f3eadbc289 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:39 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:52756
I20260812 17:48:39.142352 25279 tablet_service.cc:1514] Processing DeleteTablet for tablet 29dafadff54846aea52a77f3eadbc289 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:39 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:47452
I20260812 17:48:39.142671 25454 tablet_replica.cc:330] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811: stopping tablet replica
I20260812 17:48:39.143149 25454 raft_consensus.cc:2229] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:39.143639 25454 raft_consensus.cc:2258] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:39.143718 25456 tablet_replica.cc:330] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18: stopping tablet replica
I20260812 17:48:39.144126 25456 raft_consensus.cc:2229] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:39.144454 25456 raft_consensus.cc:2258] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:39.146095 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 23397
I20260812 17:48:39.158002 25454 ts_tablet_manager.cc:1905] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:39.158545 25456 ts_tablet_manager.cc:1905] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:39.162644 25454 ts_tablet_manager.cc:1918] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:39.162866 25454 log.cc:1196] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-0/wal/wals/29dafadff54846aea52a77f3eadbc289
I20260812 17:48:39.163375 25454 ts_tablet_manager.cc:1939] T 29dafadff54846aea52a77f3eadbc289 P b8e8d5ffa8c347f2a3ad8bc1e448d811: Deleting consensus metadata
I20260812 17:48:39.164831 25455 tablet_replica.cc:330] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c: stopping tablet replica
I20260812 17:48:39.165264 25455 raft_consensus.cc:2229] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:39.165719 25455 raft_consensus.cc:2258] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:39.170135 25456 ts_tablet_manager.cc:1918] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:39.170336 25456 log.cc:1196] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/29dafadff54846aea52a77f3eadbc289
W20260812 17:48:39.171063 24790 connection.cc:535] server connection from 127.0.0.1:59278 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 17:48:39.171468 25456 ts_tablet_manager.cc:1939] T 29dafadff54846aea52a77f3eadbc289 P f15015c53e544edeb79ea507c2cd2b18: Deleting consensus metadata
W20260812 17:48:39.172837 25006 connection.cc:441] server connection from 127.0.0.1:52756 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52756 (request call id 1) could send its response
I20260812 17:48:39.170729 25455 ts_tablet_manager.cc:1905] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:39.177358 25455 ts_tablet_manager.cc:1918] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:39.177573 25455 log.cc:1196] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/29dafadff54846aea52a77f3eadbc289
I20260812 17:48:39.178009 25455 ts_tablet_manager.cc:1939] T 29dafadff54846aea52a77f3eadbc289 P 1250825c07de44a2af1c6a4a1487de2c: Deleting consensus metadata
I20260812 17:48:39.178753 25397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:39.144863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.13.38.61:41603)
0812 17:48:39.145210 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:39.145215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:39.162773 (+ 17558us) server_negotiation.cc:407] Connection header received
0812 17:48:39.163019 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:39.163025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:39.163097 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:39.163158 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:39.166044 (+ 2886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.167008 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.168793 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.169227 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.169333 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:39.174160 (+ 4827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:39.174165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:39.174170 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:39.174208 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:39.176407 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:39.177183 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:39.177187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:39.177189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:39.177288 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:39.177632 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:39.177638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:39.177642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:39.177948 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:39.178075 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:39.178450 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 17:48:39.178585 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268}
W20260812 17:48:39.179548 25218 connection.cc:441] server connection from 127.0.0.1:47452 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47452 (request call id 1) could send its response
I20260812 17:48:39.187736 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:39.174999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33862 (local address 127.13.38.61:41603)
0812 17:48:39.175129 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:39.175135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:39.176995 (+ 1860us) server_negotiation.cc:407] Connection header received
0812 17:48:39.177186 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:39.177189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:39.177240 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:39.177293 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:39.178232 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.178992 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.180582 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.180942 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.181038 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:39.183170 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:39.183175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:39.183177 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:39.183205 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:39.185390 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:39.185988 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:39.185994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:39.185998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:39.186090 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:39.186437 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:39.186443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:39.186444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:39.186686 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:39.186798 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:39.187340 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 17:48:39.187480 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 17:48:39.191018 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:39.176022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38014 (local address 127.13.38.60:42247)
0812 17:48:39.176681 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:39.176685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:39.180089 (+ 3404us) server_negotiation.cc:407] Connection header received
0812 17:48:39.180280 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:39.180284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:39.180344 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:39.180405 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:39.181458 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.182274 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.183462 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.183868 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.183944 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:39.185864 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:39.185869 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:39.185873 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:39.185901 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:39.188785 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:39.189346 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:39.189353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:39.189357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:39.189458 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:39.189836 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:39.189839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:39.189840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:39.190194 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:39.190325 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:39.190655 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 17:48:39.190784 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":558,"spinlock_wait_cycles":1408}
I20260812 17:48:39.191267 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:39.144478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38008 (local address 127.13.38.60:42247)
0812 17:48:39.144611 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:39.144617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:39.180401 (+ 35784us) server_negotiation.cc:407] Connection header received
0812 17:48:39.180584 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:39.180587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:39.180635 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:39.180685 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:39.181605 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.182762 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.184037 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:39.184437 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:39.184532 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:39.186343 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:39.186350 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:39.186356 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:39.186387 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:39.188411 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:39.189407 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:39.189411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:39.189413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:39.189505 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:39.189832 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:39.189836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:39.189837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:39.190134 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:39.190294 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:39.190557 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 17:48:39.190706 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":43,"spinlock_wait_cycles":4480}
I20260812 17:48:40.130658 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.130039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.2:59719)
0812 17:48:40.130398 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.130514 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":98,"threads_started":1}
W20260812 17:48:40.131176 25181 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:38917 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
I20260812 17:48:40.131672 25468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.131115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.1:49211)
0812 17:48:40.131435 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.131574 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":71,"threads_started":1}
W20260812 17:48:40.132298 24962 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:38917 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
I20260812 17:48:40.132840 25472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.132392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.3:34499)
0812 17:48:40.132685 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.132759 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":76,"threads_started":1}
W20260812 17:48:40.133136 25382 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:38917 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
I20260812 17:48:40.641470 25474 raft_consensus.cc:482] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0104f86e900746688510a9947d247481)
I20260812 17:48:40.641805 25475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0104f86e900746688510a9947d247481)
I20260812 17:48:40.641785 25474 raft_consensus.cc:504] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } }
I20260812 17:48:40.641923 25475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } }
I20260812 17:48:40.642529 25475 leader_election.cc:288] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0104f86e900746688510a9947d247481 (127.13.38.62:38917), df1d07a897684999acc8fda17f1b348e (127.13.38.61:41603)
I20260812 17:48:40.642908 25474 leader_election.cc:288] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0104f86e900746688510a9947d247481 (127.13.38.62:38917), 5b41fe185daa47f696e387ee7b9f1bd5 (127.13.38.60:42247)
I20260812 17:48:40.643491 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.642710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33854)
0812 17:48:40.643263 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.643367 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":91,"threads_started":1}
I20260812 17:48:40.643685 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.643035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33860)
0812 17:48:40.643432 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.643514 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":90,"threads_started":1}
W20260812 17:48:40.643906 24373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
W20260812 17:48:40.643870 24660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
I20260812 17:48:40.645008 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.644773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33872)
0812 17:48:40.644867 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.644936 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 17:48:40.645006 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.644621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33868)
0812 17:48:40.644905 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.644950 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221}
W20260812 17:48:40.645121 24660 leader_election.cc:334] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0104f86e900746688510a9947d247481 (127.13.38.62:38917): Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
W20260812 17:48:40.645213 24373 leader_election.cc:334] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0104f86e900746688510a9947d247481 (127.13.38.62:38917): Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
I20260812 17:48:40.647778 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.643442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:41603 (local address 127.0.0.1:33876)
0812 17:48:40.643672 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.643688 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:40.643804 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:40.644085 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:40.644088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:40.644107 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:40.644261 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.644269 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.645460 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.645463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.646449 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.646460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.647315 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.647319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.647423 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.647453 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 17:48:40.647567 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 17:48:40.647654 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
I20260812 17:48:40.647871 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.643216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33876 (local address 127.13.38.61:41603)
0812 17:48:40.643816 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:40.643821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:40.643838 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:48:40.643894 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:40.643900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:40.643952 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:40.644377 (+ 425us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:40.644488 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.645308 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.646585 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.647200 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.647287 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.647341 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:40.647620 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 17:48:40.647706 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":468,"thread_start_us":78,"threads_started":1}
I20260812 17:48:40.648109 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.643193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:42247 (local address 127.0.0.1:38030)
0812 17:48:40.644021 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.644036 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:40.644105 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:40.644606 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:40.644609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:40.644638 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:40.644732 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.644737 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.645739 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.645743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.646813 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.646824 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.647635 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.647639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.647757 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.647782 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 17:48:40.647878 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 17:48:40.647962 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"thread_start_us":70,"threads_started":1}
I20260812 17:48:40.648298 24409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "df1d07a897684999acc8fda17f1b348e" is_pre_election: true
I20260812 17:48:40.648470 24409 raft_consensus.cc:2454] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b41fe185daa47f696e387ee7b9f1bd5 in term 1.
I20260812 17:48:40.648533 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.643662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38030 (local address 127.13.38.60:42247)
0812 17:48:40.644342 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:40.644348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:40.644362 (+ 14us) server_negotiation.cc:407] Connection header received
0812 17:48:40.644397 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:40.644400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:40.644437 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:40.644512 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:40.644861 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.645574 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.646982 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.647462 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.647592 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.647661 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:40.648281 (+ 620us) server_negotiation.cc:299] Negotiation successful
0812 17:48:40.648373 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":602,"thread_start_us":60,"threads_started":1}
I20260812 17:48:40.648885 24660 leader_election.cc:302] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b41fe185daa47f696e387ee7b9f1bd5, df1d07a897684999acc8fda17f1b348e; no voters: 0104f86e900746688510a9947d247481
I20260812 17:48:40.649075 25475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 17:48:40.649034 24705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "df1d07a897684999acc8fda17f1b348e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" is_pre_election: true
I20260812 17:48:40.649158 25475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0104f86e900746688510a9947d247481)
I20260812 17:48:40.649236 25475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 17:48:40.650310 25475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } }
I20260812 17:48:40.650632 24705 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate df1d07a897684999acc8fda17f1b348e in current term 2: Already voted for candidate 5b41fe185daa47f696e387ee7b9f1bd5 in this term.
I20260812 17:48:40.650735 25475 leader_election.cc:288] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [CANDIDATE]: Term 2 election: Requested vote from peers 0104f86e900746688510a9947d247481 (127.13.38.62:38917), df1d07a897684999acc8fda17f1b348e (127.13.38.61:41603)
I20260812 17:48:40.651191 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.650851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33880)
0812 17:48:40.651037 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.651109 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 17:48:40.651244 24409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "df1d07a897684999acc8fda17f1b348e"
I20260812 17:48:40.651382 24409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 1 FOLLOWER]: Advancing to term 2
I20260812 17:48:40.652000 24375 leader_election.cc:302] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df1d07a897684999acc8fda17f1b348e; no voters: 0104f86e900746688510a9947d247481, 5b41fe185daa47f696e387ee7b9f1bd5
I20260812 17:48:40.652246 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.652022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33890)
0812 17:48:40.652133 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.652185 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 17:48:40.652348 24660 leader_election.cc:334] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0104f86e900746688510a9947d247481 (127.13.38.62:38917): Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
I20260812 17:48:40.652697 24409 raft_consensus.cc:2454] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41fe185daa47f696e387ee7b9f1bd5 in term 2.
I20260812 17:48:40.652875 25474 raft_consensus.cc:2726] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 17:48:40.653055 24660 leader_election.cc:302] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b41fe185daa47f696e387ee7b9f1bd5, df1d07a897684999acc8fda17f1b348e; no voters: 0104f86e900746688510a9947d247481
I20260812 17:48:40.653251 25475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 17:48:40.653484 25475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [term 2 LEADER]: Becoming Leader. State: Replica: 5b41fe185daa47f696e387ee7b9f1bd5, State: Running, Role: LEADER
I20260812 17:48:40.653759 25475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } }
I20260812 17:48:40.654940 25488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b41fe185daa47f696e387ee7b9f1bd5. Latest consensus state: current_term: 2 leader_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } } }
I20260812 17:48:40.655040 25488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:40.655368 25491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:40.657838 25491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:40.658208 25491 catalog_manager.cc:1259] Loaded cluster ID: dfa9bbf061bc403f8452108c10365386
I20260812 17:48:40.658254 25491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:40.658752 25491 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:40.659111 25491 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: Loaded TSK: 0
I20260812 17:48:40.659857 25491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:48:40.664871 24687 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:48:40.667972 24687 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:40.670940 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.670687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33918)
0812 17:48:40.670819 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.670874 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 17:48:40.670953 24409 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [term 2 FOLLOWER]: Refusing update from remote peer 5b41fe185daa47f696e387ee7b9f1bd5: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 17:48:40.671670 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.671410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.0.0.1:33922)
0812 17:48:40.671551 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.671603 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 17:48:40.671720 25475 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 17:48:40.671904 24660 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 -> Peer 0104f86e900746688510a9947d247481 (127.13.38.62:38917): Couldn't send request to peer 0104f86e900746688510a9947d247481. Status: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 17:48:40.673527 25474 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b41fe185daa47f696e387ee7b9f1bd5. Latest consensus state: current_term: 2 leader_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } } }
I20260812 17:48:40.673710 25474 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:48:40.675966 25474 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } } }
I20260812 17:48:40.676090 25474 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1d07a897684999acc8fda17f1b348e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:48:40.676231 25493 mvcc.cc:205] Tried to move back new op lower bound from 7317737147063283712 to 7317737147000086528. Current Snapshot: MvccSnapshot[applied={T|T < 7317737147063283712}]
I20260812 17:48:40.677845 25488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0104f86e900746688510a9947d247481" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38917 } } peers { permanent_uuid: "df1d07a897684999acc8fda17f1b348e" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 41603 } attrs { promote: false } } peers { permanent_uuid: "5b41fe185daa47f696e387ee7b9f1bd5" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 42247 } attrs { promote: false } } }
I20260812 17:48:40.677968 25488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5 [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:40.687361 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.682888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52758 (local address 127.13.38.2:35863)
0812 17:48:40.683169 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:40.683175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:40.683191 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:48:40.683248 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:40.683251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:40.683303 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:40.683396 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:40.683821 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.684758 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.686085 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.686513 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.687019 (+ 506us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.687087 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:40.687122 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 17:48:40.687195 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":101,"threads_started":1}
I20260812 17:48:40.688196 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.681915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:35863 (local address 127.0.0.1:52758)
0812 17:48:40.682201 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.682217 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:40.682322 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:40.683505 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:40.683511 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:40.683556 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:40.683675 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.683681 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.684891 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.684895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.685968 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.685976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.686715 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.686720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.686829 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.686862 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 17:48:40.686917 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 17:48:40.688089 (+ 1172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":1097}
I20260812 17:48:40.688923 25082 tablet_service.cc:1467] Processing CreateTablet for tablet 06910b0288bc42d6ac8978c422c87afc (DEFAULT_TABLE table=test-table [id=97ea526772174476a9bf0e52cd008140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:40.689280 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.682552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:32929 (local address 127.0.0.1:47466)
0812 17:48:40.682915 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.682930 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:40.683022 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:40.683924 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:40.683926 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:40.683961 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:40.684043 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.684049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.686838 (+ 2789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.686840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.687947 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.687958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.688847 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.688850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.688944 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.688975 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 17:48:40.689052 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 17:48:40.689153 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282}
I20260812 17:48:40.689415 25082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06910b0288bc42d6ac8978c422c87afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:40.689816 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.682888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47466 (local address 127.13.38.3:32929)
0812 17:48:40.683687 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:40.683691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:40.683705 (+ 14us) server_negotiation.cc:407] Connection header received
0812 17:48:40.683758 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:40.683761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:40.683809 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:40.683892 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:40.685943 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.686720 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.688062 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.688729 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.688824 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.688928 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:40.689543 (+ 615us) server_negotiation.cc:299] Negotiation successful
0812 17:48:40.689634 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":95,"threads_started":1}
I20260812 17:48:40.690356 25279 tablet_service.cc:1467] Processing CreateTablet for tablet 06910b0288bc42d6ac8978c422c87afc (DEFAULT_TABLE table=test-table [id=97ea526772174476a9bf0e52cd008140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:40.690699 25279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06910b0288bc42d6ac8978c422c87afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:40.691465 25502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.682552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45161 (local address 127.0.0.1:59292)
0812 17:48:40.683366 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.683380 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:40.683475 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:40.685717 (+ 2242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:40.685721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:40.685741 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:40.685822 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.685827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.687664 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.687667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.688998 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.689010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.691086 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.691092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.691182 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.691205 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 17:48:40.691273 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 17:48:40.691342 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":541,"thread_start_us":94,"threads_started":1}
I20260812 17:48:40.691852 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.682916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.13.38.1:45161)
0812 17:48:40.685401 (+ 2485us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:40.685408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:40.685428 (+ 20us) server_negotiation.cc:407] Connection header received
0812 17:48:40.685492 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:40.685498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:40.685565 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:40.685673 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:40.685934 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.687476 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.689160 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.690815 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.690928 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.691016 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:40.691632 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 17:48:40.691721 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2379,"thread_start_us":116,"threads_started":1}
I20260812 17:48:40.692365 24862 tablet_service.cc:1467] Processing CreateTablet for tablet 06910b0288bc42d6ac8978c422c87afc (DEFAULT_TABLE table=test-table [id=97ea526772174476a9bf0e52cd008140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:40.692629 24862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06910b0288bc42d6ac8978c422c87afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:40.693367 25506 tablet_bootstrap.cc:492] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c: Bootstrap starting.
I20260812 17:48:40.693470 25505 tablet_bootstrap.cc:492] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18: Bootstrap starting.
I20260812 17:48:40.694233 25506 tablet_bootstrap.cc:654] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:40.694489 25505 tablet_bootstrap.cc:654] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:40.695266 25509 tablet_bootstrap.cc:492] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811: Bootstrap starting.
I20260812 17:48:40.695804 25506 tablet_bootstrap.cc:492] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c: No bootstrap required, opened a new log
I20260812 17:48:40.695919 25506 ts_tablet_manager.cc:1397] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:40.696389 25506 raft_consensus.cc:348] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:40.696511 25509 tablet_bootstrap.cc:654] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:40.696509 25506 raft_consensus.cc:374] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:40.696616 25506 raft_consensus.cc:729] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1250825c07de44a2af1c6a4a1487de2c, State: Initialized, Role: FOLLOWER
I20260812 17:48:40.696764 25506 consensus_queue.cc:260] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:40.697155 25506 ts_tablet_manager.cc:1428] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:40.699316 25505 tablet_bootstrap.cc:492] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18: No bootstrap required, opened a new log
I20260812 17:48:40.699450 25505 ts_tablet_manager.cc:1397] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 17:48:40.699945 25505 raft_consensus.cc:348] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:40.700073 25505 raft_consensus.cc:374] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:40.700110 25505 raft_consensus.cc:729] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f15015c53e544edeb79ea507c2cd2b18, State: Initialized, Role: FOLLOWER
I20260812 17:48:40.700250 25505 consensus_queue.cc:260] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:40.700582 25505 ts_tablet_manager.cc:1428] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:40.702286 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.701033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.2:44951)
0812 17:48:40.702106 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.702177 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":994,"thread_start_us":102,"threads_started":1}
I20260812 17:48:40.702966 25514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.698068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.3:56561)
0812 17:48:40.702786 (+ 4718us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.702852 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4625,"thread_start_us":90,"threads_started":1}
I20260812 17:48:40.703332 25177 heartbeater.cc:503] Master 127.13.38.60:42247 was elected leader, sending a full tablet report...
I20260812 17:48:40.703563 25509 tablet_bootstrap.cc:492] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811: No bootstrap required, opened a new log
I20260812 17:48:40.703668 25509 ts_tablet_manager.cc:1397] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260812 17:48:40.704213 25509 raft_consensus.cc:348] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:40.704342 25509 raft_consensus.cc:374] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:40.704385 25509 raft_consensus.cc:729] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e8d5ffa8c347f2a3ad8bc1e448d811, State: Initialized, Role: FOLLOWER
I20260812 17:48:40.704530 25379 heartbeater.cc:503] Master 127.13.38.60:42247 was elected leader, sending a full tablet report...
I20260812 17:48:40.704594 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.703565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.2:57211)
0812 17:48:40.703789 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.703840 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260812 17:48:40.704530 25509 consensus_queue.cc:260] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:40.704906 25509 ts_tablet_manager.cc:1428] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:40.706071 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.705487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.1:37397)
0812 17:48:40.705912 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.705979 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":98,"threads_started":1}
I20260812 17:48:40.706521 24961 heartbeater.cc:503] Master 127.13.38.60:42247 was elected leader, sending a full tablet report...
I20260812 17:48:40.707183 25514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.706888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.3:45333)
0812 17:48:40.707027 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.707096 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 17:48:40.708169 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.707598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.1:46463)
0812 17:48:40.707865 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.707921 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":110,"threads_started":1}
W20260812 17:48:40.723908 24967 tablet.cc:2369] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:48:40.791287 25521 raft_consensus.cc:482] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:40.791450 25521 raft_consensus.cc:504] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:40.792649 25521 leader_election.cc:288] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f15015c53e544edeb79ea507c2cd2b18 (127.13.38.2:35863), 1250825c07de44a2af1c6a4a1487de2c (127.13.38.3:32929)
I20260812 17:48:40.796806 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.793149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:55145 (local address 127.13.38.2:35863)
0812 17:48:40.793325 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:40.793331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:40.793350 (+ 19us) server_negotiation.cc:407] Connection header received
0812 17:48:40.793416 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:40.793419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:40.793469 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:40.793569 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:40.793893 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.794413 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.795629 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.796164 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.796302 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.796374 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:40.796608 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 17:48:40.796680 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 17:48:40.796850 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.793035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:35863 (local address 127.13.38.1:55145)
0812 17:48:40.793162 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.793179 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:40.793296 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:40.793596 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:40.793599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:40.793621 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:40.793762 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.793768 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.794534 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.794547 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.795418 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.795428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.796327 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.796331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.796467 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.796497 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 17:48:40.796589 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 17:48:40.796664 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 17:48:40.796983 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.793462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:42971 (local address 127.13.38.3:32929)
0812 17:48:40.793626 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:40.793633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:40.793651 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:48:40.793727 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:40.793730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:40.793773 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:40.793865 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:40.794157 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.794656 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.795764 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.796300 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.796422 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.796489 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:40.796767 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 17:48:40.796841 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 17:48:40.797435 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.793317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:32929 (local address 127.13.38.1:42971)
0812 17:48:40.793482 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.793513 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:40.793603 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:40.793879 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:40.793882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:40.793901 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:40.794035 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.794043 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.794782 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.794786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.795614 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:40.795626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.796467 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.796470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:40.796596 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.796613 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 17:48:40.796743 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 17:48:40.796817 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 17:48:40.797447 25108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06910b0288bc42d6ac8978c422c87afc" candidate_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f15015c53e544edeb79ea507c2cd2b18" is_pre_election: true
I20260812 17:48:40.797657 25108 raft_consensus.cc:2454] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e8d5ffa8c347f2a3ad8bc1e448d811 in term 0.
I20260812 17:48:40.798046 25307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06910b0288bc42d6ac8978c422c87afc" candidate_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1250825c07de44a2af1c6a4a1487de2c" is_pre_election: true
I20260812 17:48:40.798161 24792 leader_election.cc:302] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e8d5ffa8c347f2a3ad8bc1e448d811, f15015c53e544edeb79ea507c2cd2b18; no voters:
I20260812 17:48:40.798305 25307 raft_consensus.cc:2454] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e8d5ffa8c347f2a3ad8bc1e448d811 in term 0.
I20260812 17:48:40.798538 25521 raft_consensus.cc:2781] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:40.798604 25521 raft_consensus.cc:482] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:40.798657 25521 raft_consensus.cc:3037] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:40.799911 25521 raft_consensus.cc:504] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:40.800276 25521 leader_election.cc:288] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [CANDIDATE]: Term 1 election: Requested vote from peers f15015c53e544edeb79ea507c2cd2b18 (127.13.38.2:35863), 1250825c07de44a2af1c6a4a1487de2c (127.13.38.3:32929)
I20260812 17:48:40.800654 25108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06910b0288bc42d6ac8978c422c87afc" candidate_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f15015c53e544edeb79ea507c2cd2b18"
I20260812 17:48:40.800791 25108 raft_consensus.cc:3037] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:40.800808 25307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06910b0288bc42d6ac8978c422c87afc" candidate_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1250825c07de44a2af1c6a4a1487de2c"
I20260812 17:48:40.800984 25307 raft_consensus.cc:3037] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:40.801776 25108 raft_consensus.cc:2454] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e8d5ffa8c347f2a3ad8bc1e448d811 in term 1.
I20260812 17:48:40.802094 24792 leader_election.cc:302] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e8d5ffa8c347f2a3ad8bc1e448d811, f15015c53e544edeb79ea507c2cd2b18; no voters:
I20260812 17:48:40.802098 25307 raft_consensus.cc:2454] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e8d5ffa8c347f2a3ad8bc1e448d811 in term 1.
I20260812 17:48:40.802295 25521 raft_consensus.cc:2781] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:40.802627 25521 raft_consensus.cc:686] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 1 LEADER]: Becoming Leader. State: Replica: b8e8d5ffa8c347f2a3ad8bc1e448d811, State: Running, Role: LEADER
I20260812 17:48:40.803149 25521 consensus_queue.cc:237] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } }
I20260812 17:48:40.803609 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.803293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.1:45637)
0812 17:48:40.803450 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.803506 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 17:48:40.805913 24684 catalog_manager.cc:5495] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e8d5ffa8c347f2a3ad8bc1e448d811 (127.13.38.1). New cstate: current_term: 1 leader_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8e8d5ffa8c347f2a3ad8bc1e448d811" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 } health_report { overall_health: UNKNOWN } } }
I20260812 17:48:40.845726 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.842156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59296 (local address 127.13.38.1:45161)
0812 17:48:40.842302 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:40.842306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:40.842321 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:48:40.842410 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:40.842412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:40.842470 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:40.842566 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:40.842795 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.843329 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.844228 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:40.844490 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:40.844577 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:40.844822 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:40.845305 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:40.845398 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:40.845486 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 17:48:40.845538 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 17:48:40.848973 25108 raft_consensus.cc:1261] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18 [term 1 FOLLOWER]: Refusing update from remote peer b8e8d5ffa8c347f2a3ad8bc1e448d811: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:40.849211 25307 raft_consensus.cc:1261] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c [term 1 FOLLOWER]: Refusing update from remote peer b8e8d5ffa8c347f2a3ad8bc1e448d811: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:40.849660 25521 consensus_queue.cc:1035] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f15015c53e544edeb79ea507c2cd2b18" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 35863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:40.849859 25525 consensus_queue.cc:1035] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1250825c07de44a2af1c6a4a1487de2c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:40.851804 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.851458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.2:33435)
0812 17:48:40.851657 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.851716 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260812 17:48:40.852166 25514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.851835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.3:39417)
0812 17:48:40.852024 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.852090 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260812 17:48:40.855473 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.855131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.2:54443)
0812 17:48:40.855332 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.855392 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260812 17:48:40.859149 25514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.858864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.3:36699)
0812 17:48:40.859001 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.859067 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 17:48:40.860046 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.854257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.1:42859)
0812 17:48:40.859909 (+ 5652us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.859984 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2997}
I20260812 17:48:40.860852 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:40.860599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38917 (local address 127.13.38.1:44191)
0812 17:48:40.860743 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.860790 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:38917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 17:48:40.866948 24684 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:40.867143 24684 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:40.871263 24684 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b41fe185daa47f696e387ee7b9f1bd5: Sending DeleteTablet for 3 replicas of tablet 06910b0288bc42d6ac8978c422c87afc
I20260812 17:48:40.872051 25082 tablet_service.cc:1514] Processing DeleteTablet for tablet 06910b0288bc42d6ac8978c422c87afc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:40 UTC) from {username='oryx', principal='oryx/127.13.38.60@KRBTEST.COM'} at 127.0.0.1:52758
I20260812 17:48:40.872212 24862 tablet_service.cc:1514] Processing DeleteTablet for tablet 06910b0288bc42d6ac8978c422c87afc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:40 UTC) from {username='oryx', principal='oryx/127.13.38.60@KRBTEST.COM'} at 127.0.0.1:59292
I20260812 17:48:40.872435 25279 tablet_service.cc:1514] Processing DeleteTablet for tablet 06910b0288bc42d6ac8978c422c87afc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:40 UTC) from {username='oryx', principal='oryx/127.13.38.60@KRBTEST.COM'} at 127.0.0.1:47466
I20260812 17:48:40.872704 25540 tablet_replica.cc:330] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811: stopping tablet replica
I20260812 17:48:40.872835 25540 raft_consensus.cc:2229] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:40.873188 25540 raft_consensus.cc:2258] T 06910b0288bc42d6ac8978c422c87afc P b8e8d5ffa8c347f2a3ad8bc1e448d811 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:40.873400 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 24626
I20260812 17:48:40.873456 25541 tablet_replica.cc:330] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c: stopping tablet replica
I20260812 17:48:40.873600 25541 raft_consensus.cc:2229] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:40.873739 25541 raft_consensus.cc:2258] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:40.880209 25539 tablet_replica.cc:330] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18: stopping tablet replica
I20260812 17:48:40.880409 25539 raft_consensus.cc:2229] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:40.880635 25539 raft_consensus.cc:2258] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:40.882238 25539 ts_tablet_manager.cc:1905] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:40.882485 25541 ts_tablet_manager.cc:1905] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:40.884598 25539 ts_tablet_manager.cc:1918] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:40.884706 25539 log.cc:1196] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/06910b0288bc42d6ac8978c422c87afc
I20260812 17:48:40.885007 25539 ts_tablet_manager.cc:1939] T 06910b0288bc42d6ac8978c422c87afc P f15015c53e544edeb79ea507c2cd2b18: Deleting consensus metadata
I20260812 17:48:40.885841 24661 catalog_manager.cc:4841] TS f15015c53e544edeb79ea507c2cd2b18 (127.13.38.2:35863): tablet 06910b0288bc42d6ac8978c422c87afc (table test-table [id=97ea526772174476a9bf0e52cd008140]) successfully deleted
I20260812 17:48:40.885996 25541 ts_tablet_manager.cc:1918] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:40.886101 25541 log.cc:1196] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/06910b0288bc42d6ac8978c422c87afc
I20260812 17:48:40.886404 25541 ts_tablet_manager.cc:1939] T 06910b0288bc42d6ac8978c422c87afc P 1250825c07de44a2af1c6a4a1487de2c: Deleting consensus metadata
I20260812 17:48:40.887317 24661 catalog_manager.cc:4841] TS 1250825c07de44a2af1c6a4a1487de2c (127.13.38.3:32929): tablet 06910b0288bc42d6ac8978c422c87afc (table test-table [id=97ea526772174476a9bf0e52cd008140]) successfully deleted
W20260812 17:48:40.890618 25502 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:40.890065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45161 (local address 127.0.0.1:59300)
0812 17:48:40.890219 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:40.890238 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:40.890333 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:40.890479 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:45161: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 17:48:40.890853 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 24976
W20260812 17:48:40.890825 24662 catalog_manager.cc:4584] TS b8e8d5ffa8c347f2a3ad8bc1e448d811 (127.13.38.1:45161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 06910b0288bc42d6ac8978c422c87afc: Network error: Client connection negotiation failed: client connection to 127.13.38.1:45161: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260812 17:48:40.904412 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 25189
I20260812 17:48:40.922461 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 24176
I20260812 17:48:40.934502 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 24342
2026-08-12T17:48:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5832 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:41 dist-test-slave-sb2z krb5kdc[25545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:41 dist-test-slave-sb2z krb5kdc[25545](info): set up 2 sockets
Aug 12 17:48:41 dist-test-slave-sb2z krb5kdc[25545](info): commencing operation
krb5kdc: starting...
W20260812 17:48:42.337760 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.368s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:42 dist-test-slave-sb2z krb5kdc[25545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.62.keytab.
Entry for principal oryx/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.62.keytab.
I20260812 17:48:42.477046 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:42309
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.62.keytab
--principal=oryx/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:42309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:42.596860 25565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:42.597194 25565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:42.597293 25565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:42.601534 25565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:42.601634 25565 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 17:48:42.601665 25565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:42.601724 25565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:42.601778 25565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:42.601816 25565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:42.607347 25565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:42309
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.62.keytab
--principal=oryx/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:42309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:42.608536 25565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:42.609611 25565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:42.620265 25580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:42.620265 25578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:42.620528 25582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:42.620930 25565 server_base.cc:1034] running on GCE node
I20260812 17:48:42.621613 25565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:42.622910 25565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:42.624421 25565 hybrid_clock.cc:648] HybridClock initialized: now 1786556922624388 us; error 44 us; skew 500 ppm
Aug 12 17:48:42 dist-test-slave-sb2z krb5kdc[25545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556922, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:42.627580 25565 init.cc:377] Logged in from keytab as oryx/127.13.38.62@KRBTEST.COM (short username oryx)
I20260812 17:48:42.628600 25565 webserver.cc:460] Webserver started at http://127.13.38.62:45939/ using document root <none> and password file <none>
I20260812 17:48:42.629221 25565 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:42.629318 25565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:42.629570 25565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:42.631807 25565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "7e89067e90ef43cbad0dfec99f58ac3d"
format_stamp: "Formatted at 2026-08-12 17:48:42 on dist-test-slave-sb2z"
server_key: "52944570a0f1f87bf4fd42c2217fb40b"
server_key_iv: "757b1c2563776f65bd31a9635afbe287"
server_key_version: "encryptionkey@0"
I20260812 17:48:42.632431 25565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "7e89067e90ef43cbad0dfec99f58ac3d"
format_stamp: "Formatted at 2026-08-12 17:48:42 on dist-test-slave-sb2z"
server_key: "52944570a0f1f87bf4fd42c2217fb40b"
server_key_iv: "757b1c2563776f65bd31a9635afbe287"
server_key_version: "encryptionkey@0"
I20260812 17:48:42.636550 25565 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 17:48:42.639248 25592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:42.640684 25565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:48:42.640851 25565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "7e89067e90ef43cbad0dfec99f58ac3d"
format_stamp: "Formatted at 2026-08-12 17:48:42 on dist-test-slave-sb2z"
server_key: "52944570a0f1f87bf4fd42c2217fb40b"
server_key_iv: "757b1c2563776f65bd31a9635afbe287"
server_key_version: "encryptionkey@0"
I20260812 17:48:42.641005 25565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:42.665418 25565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:42.668963 25565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:42.669234 25565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:42.678797 25565 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:42309
I20260812 17:48:42.678829 25690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:42309 every 8 connection(s)
I20260812 17:48:42.679973 25565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:42.682956 25691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:42.685483 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 25565
I20260812 17:48:42.685643 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:42.685961 13464 external_mini_cluster.cc:1428] Setting key 78be6f5a8adbd251ded768e80b559e21
I20260812 17:48:42.689599 25691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d: Bootstrap starting.
I20260812 17:48:42.692022 25691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:42.692814 25691 log.cc:824] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:42.695163 25691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d: No bootstrap required, opened a new log
Aug 12 17:48:42 dist-test-slave-sb2z krb5kdc[25545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:42.700057 25691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e89067e90ef43cbad0dfec99f58ac3d" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42309 } }
I20260812 17:48:42.700398 25691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:42.700471 25691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e89067e90ef43cbad0dfec99f58ac3d, State: Initialized, Role: FOLLOWER
I20260812 17:48:42.701013 25691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e89067e90ef43cbad0dfec99f58ac3d" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42309 } }
I20260812 17:48:42.701186 25691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:42.701284 25691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:42.701390 25691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:42.701915 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:42.687598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.13.38.62:42309)
0812 17:48:42.688077 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:42.688094 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:42.688131 (+ 37us) server_negotiation.cc:407] Connection header received
0812 17:48:42.688831 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:42.688850 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:42.689193 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:42.689563 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:42.690440 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:42.691466 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:42.692471 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:42.692992 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:42.693172 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:42.696106 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:42.696113 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:42.696130 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:42.696163 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:42.698255 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:42.698787 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:42.698795 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:42.698803 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:42.698969 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:42.699315 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:42.699319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:42.699321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:42.699922 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:42.700146 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:42.700484 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 17:48:42.700807 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":149,"threads_started":1}
I20260812 17:48:42.702637 25691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e89067e90ef43cbad0dfec99f58ac3d" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42309 } }
I20260812 17:48:42.703025 25691 leader_election.cc:302] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e89067e90ef43cbad0dfec99f58ac3d; no voters:
I20260812 17:48:42.703457 25691 leader_election.cc:288] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:42.703603 25698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:42.703842 25698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [term 1 LEADER]: Becoming Leader. State: Replica: 7e89067e90ef43cbad0dfec99f58ac3d, State: Running, Role: LEADER
I20260812 17:48:42.704254 25698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e89067e90ef43cbad0dfec99f58ac3d" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42309 } }
I20260812 17:48:42.704878 25691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:42.705551 25698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e89067e90ef43cbad0dfec99f58ac3d. Latest consensus state: current_term: 1 leader_uuid: "7e89067e90ef43cbad0dfec99f58ac3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e89067e90ef43cbad0dfec99f58ac3d" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42309 } } }
I20260812 17:48:42.705694 25698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:42.706028 25699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e89067e90ef43cbad0dfec99f58ac3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e89067e90ef43cbad0dfec99f58ac3d" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42309 } } }
I20260812 17:48:42.706135 25699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:42.706205 25707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:42.709867 25707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:42.716017 25707 catalog_manager.cc:1347] Generated new cluster ID: dec4e75f233b447093db940cde4c57bc
I20260812 17:48:42.716187 25707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:42.730508 25707 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:42.731730 25707 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:42.740695 25707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d: Generated new TSK 0
I20260812 17:48:42.741552 25707 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab.
Entry for principal oryx/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab.
I20260812 17:48:42.779439 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:42309
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab
--principal=oryx/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:42.913558 25726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:42.913861 25726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:42.913911 25726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:42.918036 25726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:42.918110 25726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:42.918151 25726 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 17:48:42.918185 25726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:42.923704 25726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab
--principal=oryx/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.38.62:42309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:42.924806 25726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:42.925824 25726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:42.935884 25733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:42.935930 25734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:42.936223 25726 server_base.cc:1034] running on GCE node
W20260812 17:48:42.935927 25736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:42.936823 25726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:42.937573 25726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:42.938797 25726 hybrid_clock.cc:648] HybridClock initialized: now 1786556922938752 us; error 59 us; skew 500 ppm
Aug 12 17:48:42 dist-test-slave-sb2z krb5kdc[25545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556922, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:42.945284 25726 init.cc:377] Logged in from keytab as oryx/127.13.38.1@KRBTEST.COM (short username oryx)
I20260812 17:48:42.946420 25726 webserver.cc:460] Webserver started at http://127.13.38.1:43377/ using document root <none> and password file <none>
I20260812 17:48:42.947064 25726 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:42.947167 25726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:42.947427 25726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:42.949712 25726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "d7985e770b334cf28cb8a59799b2df2c"
format_stamp: "Formatted at 2026-08-12 17:48:42 on dist-test-slave-sb2z"
server_key: "2c0ac8c47f7d09361e9a7a69cf9a3e10"
server_key_iv: "552e143e20ad201e0dc8a48457b87d8a"
server_key_version: "encryptionkey@0"
I20260812 17:48:42.950340 25726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "d7985e770b334cf28cb8a59799b2df2c"
format_stamp: "Formatted at 2026-08-12 17:48:42 on dist-test-slave-sb2z"
server_key: "2c0ac8c47f7d09361e9a7a69cf9a3e10"
server_key_iv: "552e143e20ad201e0dc8a48457b87d8a"
server_key_version: "encryptionkey@0"
I20260812 17:48:42.954622 25726 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260812 17:48:42.957451 25746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:42.958776 25726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:42.958935 25726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "d7985e770b334cf28cb8a59799b2df2c"
format_stamp: "Formatted at 2026-08-12 17:48:42 on dist-test-slave-sb2z"
server_key: "2c0ac8c47f7d09361e9a7a69cf9a3e10"
server_key_iv: "552e143e20ad201e0dc8a48457b87d8a"
server_key_version: "encryptionkey@0"
I20260812 17:48:42.959072 25726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:42.983312 25726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:42.987115 25726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:42.987356 25726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:42.989151 25726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:42.989225 25726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:42.989283 25726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:42.989312 25726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 17:48:42 dist-test-slave-sb2z krb5kdc[25545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556922, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:43.001438 25726 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:41429
I20260812 17:48:43.001895 25927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:41429 every 8 connection(s)
I20260812 17:48:43.002563 25726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:43.005348 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:42.990241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:33979 (local address 127.13.38.62:42309)
0812 17:48:42.991559 (+ 1318us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:42.991568 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:42.991591 (+ 23us) server_negotiation.cc:407] Connection header received
0812 17:48:42.992903 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:42.992907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:42.992982 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:42.993096 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:42.994818 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:42.995786 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:42.997224 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:42.997475 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:42.997598 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.000825 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:43.000829 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:43.000831 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:43.000861 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:43.002692 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.003432 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.003438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.003440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.003603 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.003986 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.003988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.003990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.004233 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:43.004339 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.005008 (+ 669us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.005161 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 17:48:43.006297 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:42.990826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42309 (local address 127.13.38.1:33979)
0812 17:48:42.991402 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:42.991465 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:42.992598 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:42.993305 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:42.993315 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:42.993822 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:42.994625 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:42.994653 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:42.995940 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:42.995944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:42.997081 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:42.997092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:42.997664 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:42.997669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:42.997862 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:42.998570 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:42.998591 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:43.000596 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:43.002844 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.002851 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.002873 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.003274 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.003727 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.003730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.003731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.003881 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.004370 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:43.004376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:43.004664 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.004938 (+ 274us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.005301 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":140,"threads_started":1}
I20260812 17:48:43.007761 25928 heartbeater.cc:347] Connected to a master server at 127.13.38.62:42309
I20260812 17:48:43.007946 25928 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:43.008544 25928 heartbeater.cc:511] Master 127.13.38.62:42309 requested a full tablet report, sending...
I20260812 17:48:43.009276 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 25726
I20260812 17:48:43.009423 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:43.009755 13464 external_mini_cluster.cc:1428] Setting key 0620e2ee5557231c34b05043e5b0143a
I20260812 17:48:43.010537 25613 ts_manager.cc:194] Registered new tserver with Master: d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429)
I20260812 17:48:43.012276 25613 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:33979
I20260812 17:48:43.020597 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.010624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:42541 (local address 127.13.38.62:42309)
0812 17:48:43.010761 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.010766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.011036 (+ 270us) server_negotiation.cc:407] Connection header received
0812 17:48:43.011234 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.011237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.011284 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.011338 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:43.012067 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.012878 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.014455 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.014838 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.014989 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.016594 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:43.016598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:43.016600 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:43.016625 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:43.018344 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.019001 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.019004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.019006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.019103 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.019520 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.019577 (+ 57us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.019583 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.019822 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:43.019941 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.020339 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.020468 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:48:43.020749 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.010525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42309 (local address 127.13.38.1:42541)
0812 17:48:43.010947 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.010964 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.011115 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.011491 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.011494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.011793 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:43.011929 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.011937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.013326 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.013331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.014287 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.014299 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.015000 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.015008 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.015163 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.015794 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:43.015816 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:43.016407 (+ 591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:43.018498 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.018504 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.018508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.018851 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.019241 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.019246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.019249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.019375 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.019955 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:43.019959 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:43.020094 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.020244 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.020446 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":129,"threads_started":1}
WARNING: no policy specified for oryx/127.13.38.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab.
Entry for principal oryx/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab.
I20260812 17:48:43.043838 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:42309
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab
--principal=oryx/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:43.162709 25940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:43.163012 25940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:43.163097 25940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:43.167222 25940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:43.167339 25940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:43.167416 25940 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 17:48:43.167506 25940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:43.174155 25940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab
--principal=oryx/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.38.62:42309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:43.175343 25940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:43.176658 25940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:43.187150 25945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:43.187099 25949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:43.187099 25947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:43.187592 25940 server_base.cc:1034] running on GCE node
I20260812 17:48:43.187999 25940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:43.188691 25940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:43.189862 25940 hybrid_clock.cc:648] HybridClock initialized: now 1786556923189849 us; error 26 us; skew 500 ppm
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556923, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:43.192903 25940 init.cc:377] Logged in from keytab as oryx/127.13.38.2@KRBTEST.COM (short username oryx)
I20260812 17:48:43.193871 25940 webserver.cc:460] Webserver started at http://127.13.38.2:39261/ using document root <none> and password file <none>
I20260812 17:48:43.194429 25940 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:43.194491 25940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:43.194669 25940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:43.197031 25940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "aaa101680029404995cd4d34753ff23d"
format_stamp: "Formatted at 2026-08-12 17:48:43 on dist-test-slave-sb2z"
server_key: "99fe79b3c55296b9a5fd0a69b35f7d96"
server_key_iv: "749d52c28b889f6d543e60456eec1c01"
server_key_version: "encryptionkey@0"
I20260812 17:48:43.197577 25940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "aaa101680029404995cd4d34753ff23d"
format_stamp: "Formatted at 2026-08-12 17:48:43 on dist-test-slave-sb2z"
server_key: "99fe79b3c55296b9a5fd0a69b35f7d96"
server_key_iv: "749d52c28b889f6d543e60456eec1c01"
server_key_version: "encryptionkey@0"
I20260812 17:48:43.201792 25940 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 17:48:43.204368 25959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:43.205811 25940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:48:43.205955 25940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "aaa101680029404995cd4d34753ff23d"
format_stamp: "Formatted at 2026-08-12 17:48:43 on dist-test-slave-sb2z"
server_key: "99fe79b3c55296b9a5fd0a69b35f7d96"
server_key_iv: "749d52c28b889f6d543e60456eec1c01"
server_key_version: "encryptionkey@0"
I20260812 17:48:43.206080 25940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:43.227766 25940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:43.231190 25940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:43.231420 25940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:43.233170 25940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:43.233248 25940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:43.233327 25940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:43.233385 25940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556923, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:43.245710 25940 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:32951
I20260812 17:48:43.245864 26140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:32951 every 8 connection(s)
I20260812 17:48:43.246905 25940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:43.248639 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.234053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:42071 (local address 127.13.38.62:42309)
0812 17:48:43.234216 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.234221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.235389 (+ 1168us) server_negotiation.cc:407] Connection header received
0812 17:48:43.236435 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.236438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.236502 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.236593 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:43.238051 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.238708 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.240576 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.240955 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.241073 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.244489 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:43.244493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:43.244495 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:43.244525 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:43.246356 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.246906 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.246910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.246913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.247014 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.247398 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.247401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.247403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.247691 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:43.247793 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.248388 (+ 595us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.248500 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 17:48:43.249490 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.234655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42309 (local address 127.13.38.2:42071)
0812 17:48:43.235220 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.235267 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.236205 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.236917 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.236925 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.237281 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:43.237883 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.237899 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.238845 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.238848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.240447 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.240462 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.241168 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.241174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.241395 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.242153 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:43.242172 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:43.244247 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:43.246491 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.246496 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.246507 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.246774 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.247137 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.247140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.247141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.247280 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.247849 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:43.247856 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:43.248141 (+ 285us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.248298 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.248555 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":155,"threads_started":1}
I20260812 17:48:43.250720 26141 heartbeater.cc:347] Connected to a master server at 127.13.38.62:42309
I20260812 17:48:43.250880 26141 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:43.251463 26141 heartbeater.cc:511] Master 127.13.38.62:42309 requested a full tablet report, sending...
I20260812 17:48:43.252651 25614 ts_manager.cc:194] Registered new tserver with Master: aaa101680029404995cd4d34753ff23d (127.13.38.2:32951)
I20260812 17:48:43.252732 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 25940
I20260812 17:48:43.252842 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:43.253152 13464 external_mini_cluster.cc:1428] Setting key b3d45399ef78bc938fd72043997557bc
I20260812 17:48:43.253470 25614 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:42071
I20260812 17:48:43.262802 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.254085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:45527 (local address 127.13.38.62:42309)
0812 17:48:43.254208 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.254213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.255686 (+ 1473us) server_negotiation.cc:407] Connection header received
0812 17:48:43.255835 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.255837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.255875 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.255957 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:43.256420 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.256919 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.257756 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.257970 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.258079 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.259307 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:43.259311 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:43.259314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:43.259345 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:43.260827 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.261374 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.261378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.261380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.261465 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.261803 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.261807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.261809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.262099 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:43.262213 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.262543 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.262667 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 17:48:43.262807 26149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.253756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42309 (local address 127.13.38.2:45527)
0812 17:48:43.254031 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.255609 (+ 1578us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.255732 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.255978 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.255981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.256209 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:43.256303 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.256308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.257052 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.257056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.257633 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.257641 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.258096 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.258100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.258206 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.258743 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:43.258755 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:43.259166 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:43.260967 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.260972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.260974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.261262 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.261570 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.261573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.261575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.261701 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.262203 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:43.262206 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:43.262308 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.262421 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.262577 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for oryx/127.13.38.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab.
Entry for principal oryx/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab.
I20260812 17:48:43.283331 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:42309
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab
--principal=oryx/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:43.403811 26152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:43.404083 26152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:43.404163 26152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:43.408473 26152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:43.408540 26152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:43.408612 26152 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 17:48:43.408672 26152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:43.414086 26152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab
--principal=oryx/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.38.62:42309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:43.415162 26152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:43.416326 26152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:43.426214 26166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:43.426314 26165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:43.426357 26152 server_base.cc:1034] running on GCE node
W20260812 17:48:43.426249 26169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:43.426959 26152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:43.427708 26152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:43.428874 26152 hybrid_clock.cc:648] HybridClock initialized: now 1786556923428838 us; error 48 us; skew 500 ppm
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556923, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:43.431720 26152 init.cc:377] Logged in from keytab as oryx/127.13.38.3@KRBTEST.COM (short username oryx)
I20260812 17:48:43.432612 26152 webserver.cc:460] Webserver started at http://127.13.38.3:44981/ using document root <none> and password file <none>
I20260812 17:48:43.433193 26152 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:43.433285 26152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:43.433527 26152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:43.435644 26152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "9336693035da4c0b9667233bd92e8cfc"
format_stamp: "Formatted at 2026-08-12 17:48:43 on dist-test-slave-sb2z"
server_key: "f2b039260e8980386dfce7cb85542302"
server_key_iv: "005d1730170fdbb97fde38f8c3d7322c"
server_key_version: "encryptionkey@0"
I20260812 17:48:43.436249 26152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "9336693035da4c0b9667233bd92e8cfc"
format_stamp: "Formatted at 2026-08-12 17:48:43 on dist-test-slave-sb2z"
server_key: "f2b039260e8980386dfce7cb85542302"
server_key_iv: "005d1730170fdbb97fde38f8c3d7322c"
server_key_version: "encryptionkey@0"
I20260812 17:48:43.440140 26152 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:48:43.442706 26178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:43.444182 26152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:43.444352 26152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "9336693035da4c0b9667233bd92e8cfc"
format_stamp: "Formatted at 2026-08-12 17:48:43 on dist-test-slave-sb2z"
server_key: "f2b039260e8980386dfce7cb85542302"
server_key_iv: "005d1730170fdbb97fde38f8c3d7322c"
server_key_version: "encryptionkey@0"
I20260812 17:48:43.444481 26152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:43.467964 26152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:43.471164 26152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:43.471393 26152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:43.472997 26152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:43.473091 26152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:43.473166 26152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:43.473214 26152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556923, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:43.484783 26152 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:36135
I20260812 17:48:43.484825 26373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:36135 every 8 connection(s)
I20260812 17:48:43.485996 26152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:43.488611 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.474105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:37431 (local address 127.13.38.62:42309)
0812 17:48:43.474242 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.474246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.475333 (+ 1087us) server_negotiation.cc:407] Connection header received
0812 17:48:43.476494 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.476497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.476558 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.476656 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:43.478459 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.479383 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.481355 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.481677 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.481770 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.484713 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:43.484718 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:43.484722 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:43.484751 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:43.486369 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.487003 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.487006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.487008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.487117 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.487580 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.487582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.487584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.487783 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:43.487862 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.488383 (+ 521us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.488488 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 17:48:43.489442 26194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.474669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42309 (local address 127.13.38.3:37431)
0812 17:48:43.475167 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.475212 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.476234 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.476907 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.476919 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.477472 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:43.478255 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.478282 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.479570 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.479574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.481185 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.481200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.481860 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.481867 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.482102 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.482879 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:43.482899 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:43.484528 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:43.486499 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.486505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.486521 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.486860 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.487242 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.487247 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.487251 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.487438 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.487878 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:43.487884 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:43.488129 (+ 245us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.488287 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.488569 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":134,"threads_started":1}
I20260812 17:48:43.490862 26375 heartbeater.cc:347] Connected to a master server at 127.13.38.62:42309
I20260812 17:48:43.491099 26375 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:43.491376 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 26152
I20260812 17:48:43.491511 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:43.491813 26375 heartbeater.cc:511] Master 127.13.38.62:42309 requested a full tablet report, sending...
I20260812 17:48:43.491930 13464 external_mini_cluster.cc:1428] Setting key d89a130c24a3aa1247d6cde1af7e0928
I20260812 17:48:43.493058 25613 ts_manager.cc:194] Registered new tserver with Master: 9336693035da4c0b9667233bd92e8cfc (127.13.38.3:36135)
I20260812 17:48:43.493832 25613 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:37431
I20260812 17:48:43.496145 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:48:43.501905 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.493427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42309 (local address 127.13.38.3:48237)
0812 17:48:43.493772 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.493788 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.493899 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.494214 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.494217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.494446 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:43.494583 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.494591 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.495723 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.495727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.496440 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.496451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.496947 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.496951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.497075 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.497664 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:43.497685 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:43.498192 (+ 507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:43.500363 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.500366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.500368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.500583 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.500908 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.500910 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.500912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.500990 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.501458 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:43.501460 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:43.501541 (+ 81us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.501635 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.501757 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":122,"threads_started":1}
I20260812 17:48:43.502295 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.493552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:48237 (local address 127.13.38.62:42309)
0812 17:48:43.493686 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.493693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.493880 (+ 187us) server_negotiation.cc:407] Connection header received
0812 17:48:43.493993 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.493998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.494066 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.494160 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:43.494703 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.495563 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.496576 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.496818 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.496911 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.498350 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:43.498353 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:43.498355 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:43.498379 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:43.500229 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.500690 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.500697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.500701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.500798 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.501083 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.501089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.501093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.501357 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:43.501467 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.501727 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.501869 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556923, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 17:48:43.525624 25696 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:43.518623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43750 (local address 127.13.38.62:42309)
0812 17:48:43.518799 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.518808 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.518823 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:48:43.518870 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.518872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.518932 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.519012 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:43.519837 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.520475 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.521379 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.521647 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.521747 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.525458 (+ 3711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43750: BlockingRecv error: recv got EOF from 127.0.0.1:43750 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556923, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:43.535281 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.527314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43758 (local address 127.13.38.62:42309)
0812 17:48:43.527453 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.527458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.527664 (+ 206us) server_negotiation.cc:407] Connection header received
0812 17:48:43.527766 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.527768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.527811 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.527879 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:43.528476 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.528988 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.529769 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.529971 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.530044 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.532449 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:43.532452 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:43.532454 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:43.532482 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:43.533758 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.534157 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.534160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.534161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.534236 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.534466 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.534468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.534469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.534646 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:43.534751 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.534951 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.535057 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 17:48:43.538498 25613 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:48:43.541738 25613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:43.557430 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.551622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39060 (local address 127.13.38.3:36135)
0812 17:48:43.552420 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.552435 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.552461 (+ 26us) server_negotiation.cc:407] Connection header received
0812 17:48:43.552551 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.552560 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.552785 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.552962 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:43.553257 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.554205 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.555846 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.556642 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.556754 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.556923 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.557145 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.557262 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":100,"threads_started":1}
I20260812 17:48:43.557451 26408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.551379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:36135 (local address 127.0.0.1:39060)
0812 17:48:43.551900 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.551969 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.552116 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.552959 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.552965 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.552988 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:43.553137 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.553144 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.554324 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.554328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.555707 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.555719 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.556784 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.556788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.556917 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.556966 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.557088 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.557256 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":117,"spinlock_wait_cycles":1920,"thread_start_us":134,"threads_started":1}
I20260812 17:48:43.557817 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.551324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.13.38.1:41429)
0812 17:48:43.552554 (+ 1230us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.552562 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.552588 (+ 26us) server_negotiation.cc:407] Connection header received
0812 17:48:43.552674 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.552679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.552883 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.553344 (+ 461us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:43.553507 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.554671 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.556116 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.556771 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.557352 (+ 581us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.557508 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.557564 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.557663 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1122,"thread_start_us":109,"threads_started":1}
I20260812 17:48:43.557929 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.551168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:41429 (local address 127.0.0.1:36834)
0812 17:48:43.551953 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.551969 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.552116 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.553056 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.553059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.553081 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:43.553195 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.553201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.554803 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.554806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.555985 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.555994 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.556904 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.556907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.557005 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.557022 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.557068 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.557256 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":43,"spinlock_wait_cycles":12928,"thread_start_us":109,"threads_started":1}
I20260812 17:48:43.558492 26411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.552164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:32951 (local address 127.0.0.1:54572)
0812 17:48:43.552508 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.552522 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.552606 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.553471 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.553474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.553494 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:43.553593 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.553600 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.554979 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.554982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.556206 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.556217 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.557479 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.557485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.558170 (+ 685us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.558197 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.558281 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.558357 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":25,"thread_start_us":105,"threads_started":1}
I20260812 17:48:43.558627 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.552507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54572 (local address 127.13.38.2:32951)
0812 17:48:43.553021 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.553035 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.553056 (+ 21us) server_negotiation.cc:407] Connection header received
0812 17:48:43.553125 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.553130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.553318 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.553472 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:43.553726 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.554801 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.556340 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.557155 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.557257 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.557407 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.558350 (+ 943us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.558455 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":117,"threads_started":1}
I20260812 17:48:43.560746 26264 tablet_service.cc:1467] Processing CreateTablet for tablet d7fd5995173b472581daf46869204fb1 (DEFAULT_TABLE table=test-table [id=c70c05f312804075a3ad895fcb385aea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:43.561082 26041 tablet_service.cc:1467] Processing CreateTablet for tablet d7fd5995173b472581daf46869204fb1 (DEFAULT_TABLE table=test-table [id=c70c05f312804075a3ad895fcb385aea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:43.561854 26264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7fd5995173b472581daf46869204fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:43.562039 26041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7fd5995173b472581daf46869204fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:43.568909 26415 tablet_bootstrap.cc:492] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc: Bootstrap starting.
I20260812 17:48:43.570382 25823 tablet_service.cc:1467] Processing CreateTablet for tablet d7fd5995173b472581daf46869204fb1 (DEFAULT_TABLE table=test-table [id=c70c05f312804075a3ad895fcb385aea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:43.571179 26415 tablet_bootstrap.cc:654] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:43.571401 25823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7fd5995173b472581daf46869204fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:43.572145 26416 tablet_bootstrap.cc:492] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d: Bootstrap starting.
I20260812 17:48:43.572249 26415 log.cc:824] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:43.574496 26416 tablet_bootstrap.cc:654] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:43.574930 26415 tablet_bootstrap.cc:492] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc: No bootstrap required, opened a new log
I20260812 17:48:43.575150 26415 ts_tablet_manager.cc:1397] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:48:43.575381 26416 log.cc:824] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:43.577589 26416 tablet_bootstrap.cc:492] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d: No bootstrap required, opened a new log
I20260812 17:48:43.577786 26416 ts_tablet_manager.cc:1397] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:48:43.578132 26420 tablet_bootstrap.cc:492] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c: Bootstrap starting.
I20260812 17:48:43.579205 26415 raft_consensus.cc:348] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } }
I20260812 17:48:43.579489 26415 raft_consensus.cc:374] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:43.579573 26415 raft_consensus.cc:729] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336693035da4c0b9667233bd92e8cfc, State: Initialized, Role: FOLLOWER
I20260812 17:48:43.580081 26415 consensus_queue.cc:260] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } }
I20260812 17:48:43.580391 26420 tablet_bootstrap.cc:654] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:43.581318 26420 log.cc:824] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:43.581615 26415 ts_tablet_manager.cc:1428] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 17:48:43.581720 26375 heartbeater.cc:503] Master 127.13.38.62:42309 was elected leader, sending a full tablet report...
I20260812 17:48:43.582816 26416 raft_consensus.cc:348] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } }
I20260812 17:48:43.583115 26416 raft_consensus.cc:374] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:43.583177 26416 raft_consensus.cc:729] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaa101680029404995cd4d34753ff23d, State: Initialized, Role: FOLLOWER
I20260812 17:48:43.583302 26420 tablet_bootstrap.cc:492] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c: No bootstrap required, opened a new log
I20260812 17:48:43.583504 26420 ts_tablet_manager.cc:1397] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:48:43.583693 26416 consensus_queue.cc:260] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } }
I20260812 17:48:43.584784 26416 ts_tablet_manager.cc:1428] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 17:48:43.584867 26141 heartbeater.cc:503] Master 127.13.38.62:42309 was elected leader, sending a full tablet report...
I20260812 17:48:43.587745 26420 raft_consensus.cc:348] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } }
I20260812 17:48:43.588016 26420 raft_consensus.cc:374] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:43.588101 26420 raft_consensus.cc:729] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7985e770b334cf28cb8a59799b2df2c, State: Initialized, Role: FOLLOWER
I20260812 17:48:43.588550 26420 consensus_queue.cc:260] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } }
I20260812 17:48:43.589250 25928 heartbeater.cc:503] Master 127.13.38.62:42309 was elected leader, sending a full tablet report...
I20260812 17:48:43.589535 26420 ts_tablet_manager.cc:1428] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 17:48:43.594858 26425 raft_consensus.cc:482] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:43.595068 26425 raft_consensus.cc:504] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } }
I20260812 17:48:43.596436 26425 leader_election.cc:288] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429), 9336693035da4c0b9667233bd92e8cfc (127.13.38.3:36135)
I20260812 17:48:43.601445 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.596937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:49039 (local address 127.13.38.3:36135)
0812 17:48:43.597093 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.597098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.597111 (+ 13us) server_negotiation.cc:407] Connection header received
0812 17:48:43.597167 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.597169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.597248 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.597333 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:43.597807 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.598633 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.600198 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.600832 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.600934 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.600991 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.601271 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.601335 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 17:48:43.601468 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.596708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:36135 (local address 127.13.38.2:49039)
0812 17:48:43.596821 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.596838 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.596966 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.597399 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.597401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.597505 (+ 104us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:43.597670 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.597682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.598747 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.598751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.600053 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.600064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.600966 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.600970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.601120 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.601148 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.601235 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.601314 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"spinlock_wait_cycles":19328}
I20260812 17:48:43.602100 26428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.596714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:41429 (local address 127.13.38.2:54037)
0812 17:48:43.597348 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.597364 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.597477 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.597750 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.597754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.597775 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:43.598399 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.598411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.599455 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.599459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.600636 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.600648 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.601595 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.601599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.601736 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.601760 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.601874 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.601949 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":273,"thread_start_us":118,"threads_started":1}
I20260812 17:48:43.602114 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.596764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:54037 (local address 127.13.38.1:41429)
0812 17:48:43.596894 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.596899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.597473 (+ 574us) server_negotiation.cc:407] Connection header received
0812 17:48:43.597566 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.597570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.597623 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.597717 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:43.598567 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.599291 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.600802 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.601443 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.601579 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.601649 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.601924 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.602022 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 17:48:43.602319 26299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7fd5995173b472581daf46869204fb1" candidate_uuid: "aaa101680029404995cd4d34753ff23d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336693035da4c0b9667233bd92e8cfc" is_pre_election: true
I20260812 17:48:43.602700 26299 raft_consensus.cc:2454] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaa101680029404995cd4d34753ff23d in term 0.
I20260812 17:48:43.602993 25857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7fd5995173b472581daf46869204fb1" candidate_uuid: "aaa101680029404995cd4d34753ff23d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7985e770b334cf28cb8a59799b2df2c" is_pre_election: true
I20260812 17:48:43.603322 25857 raft_consensus.cc:2454] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaa101680029404995cd4d34753ff23d in term 0.
I20260812 17:48:43.603313 25964 leader_election.cc:302] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9336693035da4c0b9667233bd92e8cfc, aaa101680029404995cd4d34753ff23d; no voters:
I20260812 17:48:43.603649 26425 raft_consensus.cc:2781] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:43.603739 26425 raft_consensus.cc:482] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:43.603771 26425 raft_consensus.cc:3037] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:43.604936 26425 raft_consensus.cc:504] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } }
I20260812 17:48:43.605291 26425 leader_election.cc:288] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [CANDIDATE]: Term 1 election: Requested vote from peers d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429), 9336693035da4c0b9667233bd92e8cfc (127.13.38.3:36135)
I20260812 17:48:43.605731 26299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7fd5995173b472581daf46869204fb1" candidate_uuid: "aaa101680029404995cd4d34753ff23d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336693035da4c0b9667233bd92e8cfc"
I20260812 17:48:43.605748 25857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7fd5995173b472581daf46869204fb1" candidate_uuid: "aaa101680029404995cd4d34753ff23d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7985e770b334cf28cb8a59799b2df2c"
I20260812 17:48:43.605870 26299 raft_consensus.cc:3037] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:43.605906 25857 raft_consensus.cc:3037] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:43.607080 25857 raft_consensus.cc:2454] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaa101680029404995cd4d34753ff23d in term 1.
I20260812 17:48:43.607107 26299 raft_consensus.cc:2454] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaa101680029404995cd4d34753ff23d in term 1.
I20260812 17:48:43.607553 25963 leader_election.cc:302] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaa101680029404995cd4d34753ff23d, d7985e770b334cf28cb8a59799b2df2c; no voters:
I20260812 17:48:43.607844 26425 raft_consensus.cc:2781] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:43.608124 26425 raft_consensus.cc:686] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 1 LEADER]: Becoming Leader. State: Replica: aaa101680029404995cd4d34753ff23d, State: Running, Role: LEADER
I20260812 17:48:43.608557 26425 consensus_queue.cc:237] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } }
I20260812 17:48:43.612287 25614 catalog_manager.cc:5495] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d reported cstate change: term changed from 0 to 1, leader changed from <none> to aaa101680029404995cd4d34753ff23d (127.13.38.2). New cstate: current_term: 1 leader_uuid: "aaa101680029404995cd4d34753ff23d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } health_report { overall_health: HEALTHY } } }
I20260812 17:48:43.638037 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.634173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54574 (local address 127.13.38.2:32951)
0812 17:48:43.634316 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.634323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.634340 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:48:43.634544 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.634559 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.634622 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.634754 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:43.635031 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.635575 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.636455 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.636697 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.636813 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.637125 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:43.637598 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:43.637736 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.637799 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.637858 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 17:48:43.644431 25857 raft_consensus.cc:1261] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Refusing update from remote peer aaa101680029404995cd4d34753ff23d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:43.644771 26299 raft_consensus.cc:1261] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Refusing update from remote peer aaa101680029404995cd4d34753ff23d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:43.645291 26425 consensus_queue.cc:1035] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:43.645507 26429 consensus_queue.cc:1035] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:43.658425 26434 mvcc.cc:205] Tried to move back new op lower bound from 7317737159239864320 to 7317737159103918080. Current Snapshot: MvccSnapshot[applied={T|T < 7317737159239864320}]
I20260812 17:48:43.662879 25614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:43.663110 25614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:43.665663 25614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d: Sending DeleteTablet for 3 replicas of tablet d7fd5995173b472581daf46869204fb1
I20260812 17:48:43.666566 25823 tablet_service.cc:1514] Processing DeleteTablet for tablet d7fd5995173b472581daf46869204fb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:43 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:36834
I20260812 17:48:43.666558 26041 tablet_service.cc:1514] Processing DeleteTablet for tablet d7fd5995173b472581daf46869204fb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:43 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:54572
I20260812 17:48:43.667421 26448 tablet_replica.cc:330] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d: stopping tablet replica
I20260812 17:48:43.667407 26264 tablet_service.cc:1514] Processing DeleteTablet for tablet d7fd5995173b472581daf46869204fb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:43 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:39060
I20260812 17:48:43.667457 25614 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 17:48:43.667855 26443 tablet_replica.cc:330] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c: stopping tablet replica
W20260812 17:48:43.667919 25614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:43.668077 26450 tablet_replica.cc:330] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc: stopping tablet replica
I20260812 17:48:43.668430 26450 raft_consensus.cc:2229] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:43.668742 26448 raft_consensus.cc:2229] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:43.669183 26448 raft_consensus.cc:2258] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:43.669376 26450 raft_consensus.cc:2258] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:43.668458 26443 raft_consensus.cc:2229] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:43.669723 26443 raft_consensus.cc:2258] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:43.671876 26450 ts_tablet_manager.cc:1905] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:43.671876 26443 ts_tablet_manager.cc:1905] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:43.674669 25823 tablet_service.cc:1467] Processing CreateTablet for tablet 2245ca7dd1434e838889597d11c4e4ca (DEFAULT_TABLE table=test-table [id=2fc60c5b114545c19aa7489ea3dc8fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:43.674679 26264 tablet_service.cc:1467] Processing CreateTablet for tablet 2245ca7dd1434e838889597d11c4e4ca (DEFAULT_TABLE table=test-table [id=2fc60c5b114545c19aa7489ea3dc8fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:43.674978 26264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2245ca7dd1434e838889597d11c4e4ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:43.675505 26450 ts_tablet_manager.cc:1918] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:43.675669 26450 log.cc:1196] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/d7fd5995173b472581daf46869204fb1
I20260812 17:48:43.675871 26041 tablet_service.cc:1467] Processing CreateTablet for tablet 2245ca7dd1434e838889597d11c4e4ca (DEFAULT_TABLE table=test-table [id=2fc60c5b114545c19aa7489ea3dc8fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:43.676015 26450 ts_tablet_manager.cc:1939] T d7fd5995173b472581daf46869204fb1 P 9336693035da4c0b9667233bd92e8cfc: Deleting consensus metadata
I20260812 17:48:43.676226 26041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2245ca7dd1434e838889597d11c4e4ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:43.677343 26415 tablet_bootstrap.cc:492] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc: Bootstrap starting.
I20260812 17:48:43.678324 26415 tablet_bootstrap.cc:654] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:43.679003 26416 tablet_bootstrap.cc:492] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d: Bootstrap starting.
I20260812 17:48:43.679704 26448 ts_tablet_manager.cc:1905] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:43.680094 26416 tablet_bootstrap.cc:654] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:43.681152 25596 catalog_manager.cc:4841] TS 9336693035da4c0b9667233bd92e8cfc (127.13.38.3:36135): tablet d7fd5995173b472581daf46869204fb1 (table test-table [id=c70c05f312804075a3ad895fcb385aea]) successfully deleted
I20260812 17:48:43.682261 25823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2245ca7dd1434e838889597d11c4e4ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:43.682476 26415 tablet_bootstrap.cc:492] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc: No bootstrap required, opened a new log
I20260812 17:48:43.682577 26415 ts_tablet_manager.cc:1397] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 17:48:43.683084 26415 raft_consensus.cc:348] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.683221 26415 raft_consensus.cc:374] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:43.683259 26415 raft_consensus.cc:729] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336693035da4c0b9667233bd92e8cfc, State: Initialized, Role: FOLLOWER
I20260812 17:48:43.683380 26415 consensus_queue.cc:260] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.684500 26415 ts_tablet_manager.cc:1428] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 17:48:43.685184 26420 tablet_bootstrap.cc:492] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c: Bootstrap starting.
I20260812 17:48:43.685794 26416 tablet_bootstrap.cc:492] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d: No bootstrap required, opened a new log
I20260812 17:48:43.685901 26416 ts_tablet_manager.cc:1397] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260812 17:48:43.686333 26420 tablet_bootstrap.cc:654] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:43.686398 26416 raft_consensus.cc:348] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.686525 26416 raft_consensus.cc:374] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:43.686564 26416 raft_consensus.cc:729] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaa101680029404995cd4d34753ff23d, State: Initialized, Role: FOLLOWER
I20260812 17:48:43.686704 26416 consensus_queue.cc:260] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.686973 26416 ts_tablet_manager.cc:1428] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 17:48:43.687510 26448 ts_tablet_manager.cc:1918] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:43.688068 26443 ts_tablet_manager.cc:1918] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:43.688235 26448 log.cc:1196] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/d7fd5995173b472581daf46869204fb1
I20260812 17:48:43.688361 26443 log.cc:1196] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/wal/wals/d7fd5995173b472581daf46869204fb1
I20260812 17:48:43.688778 26448 ts_tablet_manager.cc:1939] T d7fd5995173b472581daf46869204fb1 P aaa101680029404995cd4d34753ff23d: Deleting consensus metadata
I20260812 17:48:43.689206 26443 ts_tablet_manager.cc:1939] T d7fd5995173b472581daf46869204fb1 P d7985e770b334cf28cb8a59799b2df2c: Deleting consensus metadata
I20260812 17:48:43.691017 25595 catalog_manager.cc:4841] TS aaa101680029404995cd4d34753ff23d (127.13.38.2:32951): tablet d7fd5995173b472581daf46869204fb1 (table test-table [id=c70c05f312804075a3ad895fcb385aea]) successfully deleted
I20260812 17:48:43.692008 25595 catalog_manager.cc:4841] TS d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429): tablet d7fd5995173b472581daf46869204fb1 (table test-table [id=c70c05f312804075a3ad895fcb385aea]) successfully deleted
I20260812 17:48:43.692426 26420 tablet_bootstrap.cc:492] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c: No bootstrap required, opened a new log
I20260812 17:48:43.692555 26420 ts_tablet_manager.cc:1397] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260812 17:48:43.693259 26420 raft_consensus.cc:348] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.693387 26420 raft_consensus.cc:374] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:43.693434 26420 raft_consensus.cc:729] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7985e770b334cf28cb8a59799b2df2c, State: Initialized, Role: FOLLOWER
I20260812 17:48:43.693576 26420 consensus_queue.cc:260] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.693857 26420 ts_tablet_manager.cc:1428] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 17:48:43.717083 26426 raft_consensus.cc:482] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:43.717300 26426 raft_consensus.cc:504] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.718526 26426 leader_election.cc:288] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aaa101680029404995cd4d34753ff23d (127.13.38.2:32951), 9336693035da4c0b9667233bd92e8cfc (127.13.38.3:36135)
I20260812 17:48:43.723565 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.718771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:32951 (local address 127.13.38.1:59409)
0812 17:48:43.719456 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.719476 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.719624 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.720026 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.720028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.720044 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:43.720139 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.720143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.720872 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.720876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.721989 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.722003 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.723048 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.723051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.723170 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.723201 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.723293 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.723366 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":326,"thread_start_us":105,"threads_started":1}
I20260812 17:48:43.723517 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.718952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:59409 (local address 127.13.38.2:32951)
0812 17:48:43.719081 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.719086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.719583 (+ 497us) server_negotiation.cc:407] Connection header received
0812 17:48:43.719756 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.719759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.719809 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.719865 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:43.720255 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.720747 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.722140 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.722755 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.722837 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.722919 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.723322 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.723403 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 17:48:43.723608 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.718931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:49987 (local address 127.13.38.3:36135)
0812 17:48:43.719083 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.719088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.719239 (+ 151us) server_negotiation.cc:407] Connection header received
0812 17:48:43.719494 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.719497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.719576 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.719669 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:43.719998 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.720769 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.722372 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.722955 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.723067 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.723137 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.723358 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.723445 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 17:48:43.724140 26458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.718768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:36135 (local address 127.13.38.1:49987)
0812 17:48:43.719130 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.719150 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.719290 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.719697 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.719702 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.719724 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:43.719877 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.719884 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.720980 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.720985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.722177 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.722189 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.723087 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.723091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.723197 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.723225 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.723882 (+ 657us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.723971 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":92,"threads_started":1}
I20260812 17:48:43.724606 26299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2245ca7dd1434e838889597d11c4e4ca" candidate_uuid: "d7985e770b334cf28cb8a59799b2df2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336693035da4c0b9667233bd92e8cfc" is_pre_election: true
I20260812 17:48:43.724606 26068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2245ca7dd1434e838889597d11c4e4ca" candidate_uuid: "d7985e770b334cf28cb8a59799b2df2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaa101680029404995cd4d34753ff23d" is_pre_election: true
I20260812 17:48:43.724787 26299 raft_consensus.cc:2454] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7985e770b334cf28cb8a59799b2df2c in term 0.
I20260812 17:48:43.724849 26068 raft_consensus.cc:2454] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7985e770b334cf28cb8a59799b2df2c in term 0.
I20260812 17:48:43.725216 25752 leader_election.cc:302] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9336693035da4c0b9667233bd92e8cfc, d7985e770b334cf28cb8a59799b2df2c; no voters:
I20260812 17:48:43.725569 26426 raft_consensus.cc:2781] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:43.725633 26426 raft_consensus.cc:482] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:43.725682 26426 raft_consensus.cc:3037] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:43.726926 26426 raft_consensus.cc:504] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.727348 26426 leader_election.cc:288] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [CANDIDATE]: Term 1 election: Requested vote from peers aaa101680029404995cd4d34753ff23d (127.13.38.2:32951), 9336693035da4c0b9667233bd92e8cfc (127.13.38.3:36135)
I20260812 17:48:43.727767 26299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2245ca7dd1434e838889597d11c4e4ca" candidate_uuid: "d7985e770b334cf28cb8a59799b2df2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336693035da4c0b9667233bd92e8cfc"
I20260812 17:48:43.727767 26068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2245ca7dd1434e838889597d11c4e4ca" candidate_uuid: "d7985e770b334cf28cb8a59799b2df2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaa101680029404995cd4d34753ff23d"
I20260812 17:48:43.727916 26299 raft_consensus.cc:3037] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:43.727902 26068 raft_consensus.cc:3037] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:43.729171 26299 raft_consensus.cc:2454] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7985e770b334cf28cb8a59799b2df2c in term 1.
I20260812 17:48:43.729172 26068 raft_consensus.cc:2454] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7985e770b334cf28cb8a59799b2df2c in term 1.
I20260812 17:48:43.729513 25748 leader_election.cc:302] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaa101680029404995cd4d34753ff23d, d7985e770b334cf28cb8a59799b2df2c; no voters:
I20260812 17:48:43.729717 26426 raft_consensus.cc:2781] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:43.729995 26426 raft_consensus.cc:686] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 1 LEADER]: Becoming Leader. State: Replica: d7985e770b334cf28cb8a59799b2df2c, State: Running, Role: LEADER
I20260812 17:48:43.730279 26426 consensus_queue.cc:237] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.733134 25613 catalog_manager.cc:5495] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c reported cstate change: term changed from 0 to 1, leader changed from <none> to d7985e770b334cf28cb8a59799b2df2c (127.13.38.1). New cstate: current_term: 1 leader_uuid: "d7985e770b334cf28cb8a59799b2df2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } health_report { overall_health: UNKNOWN } } }
W20260812 17:48:43.737067 26378 tablet.cc:2369] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:48:43.748224 26142 tablet.cc:2369] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:48:43.754196 25929 tablet.cc:2369] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556922, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.62@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:43.779727 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.769371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42309 (local address 127.0.0.1:43764)
0812 17:48:43.769801 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.769818 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.769973 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.770306 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.770311 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.770978 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:43.771113 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.771123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.772116 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.772120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.773074 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.773086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.773606 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.773610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.773708 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.774385 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:43.774402 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:43.776205 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:43.778014 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.778025 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.778032 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.778279 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.778563 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:43.778569 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:43.778574 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:43.778660 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:43.779063 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:43.779072 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:43.779188 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.779307 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.779482 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":131,"threads_started":1}
I20260812 17:48:43.779727 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.769516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43764 (local address 127.13.38.62:42309)
0812 17:48:43.769655 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.769660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.769909 (+ 249us) server_negotiation.cc:407] Connection header received
0812 17:48:43.770113 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.770116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.770159 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.770223 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:43.771275 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.771979 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.773239 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.773472 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.773570 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.776364 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:43.776367 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:43.776369 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:43.776391 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:43.777882 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.778385 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.778390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.778391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.778471 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:43.778750 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:43.778754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:43.778756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:43.778942 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:43.779053 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.779367 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.779598 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":56}
I20260812 17:48:43.783998 25613 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:43764:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 17:48:43.784792 25613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:43.790432 26041 tablet_service.cc:1467] Processing CreateTablet for tablet c1c115ddab4b49e798906b45d1f9f893 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3bb2c3e0bb1f408a976ef86411021075]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 17:48:43.790534 25823 tablet_service.cc:1467] Processing CreateTablet for tablet c1c115ddab4b49e798906b45d1f9f893 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3bb2c3e0bb1f408a976ef86411021075]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 17:48:43.790973 26041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1c115ddab4b49e798906b45d1f9f893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:43.791021 25823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1c115ddab4b49e798906b45d1f9f893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:43.792289 26264 tablet_service.cc:1467] Processing CreateTablet for tablet c1c115ddab4b49e798906b45d1f9f893 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3bb2c3e0bb1f408a976ef86411021075]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 17:48:43.792694 26264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1c115ddab4b49e798906b45d1f9f893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:43.794224 26416 tablet_bootstrap.cc:492] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d: Bootstrap starting.
I20260812 17:48:43.795320 26416 tablet_bootstrap.cc:654] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:43.795573 26415 tablet_bootstrap.cc:492] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: Bootstrap starting.
I20260812 17:48:43.797026 26415 tablet_bootstrap.cc:654] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:43.797448 26416 tablet_bootstrap.cc:492] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d: No bootstrap required, opened a new log
I20260812 17:48:43.797551 26416 ts_tablet_manager.cc:1397] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 17:48:43.798184 26415 tablet_bootstrap.cc:492] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: No bootstrap required, opened a new log
I20260812 17:48:43.798301 26415 ts_tablet_manager.cc:1397] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:43.798367 26416 raft_consensus.cc:348] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.798514 26416 raft_consensus.cc:374] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:43.798561 26416 raft_consensus.cc:729] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaa101680029404995cd4d34753ff23d, State: Initialized, Role: FOLLOWER
I20260812 17:48:43.798712 26416 consensus_queue.cc:260] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.798870 26415 raft_consensus.cc:348] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.798995 26415 raft_consensus.cc:374] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:43.799027 26415 raft_consensus.cc:729] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336693035da4c0b9667233bd92e8cfc, State: Initialized, Role: FOLLOWER
I20260812 17:48:43.798998 26429 tablet_replica.cc:439] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } }
I20260812 17:48:43.799160 26429 tablet_replica.cc:442] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:43.799157 26415 consensus_queue.cc:260] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.799417 26416 ts_tablet_manager.cc:1428] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 17:48:43.799422 26415 ts_tablet_manager.cc:1428] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:43.799705 26421 tablet_replica.cc:439] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } }
I20260812 17:48:43.799868 26421 tablet_replica.cc:442] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:43.799857 26420 tablet_bootstrap.cc:492] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c: Bootstrap starting.
I20260812 17:48:43.801012 26420 tablet_bootstrap.cc:654] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:43.802856 26420 tablet_bootstrap.cc:492] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c: No bootstrap required, opened a new log
I20260812 17:48:43.802990 26420 ts_tablet_manager.cc:1397] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:43.803484 26420 raft_consensus.cc:348] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.803661 26420 raft_consensus.cc:374] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:43.803710 26420 raft_consensus.cc:729] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7985e770b334cf28cb8a59799b2df2c, State: Initialized, Role: FOLLOWER
I20260812 17:48:43.804023 26420 consensus_queue.cc:260] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.804363 26420 ts_tablet_manager.cc:1428] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 17:48:43.804325 26426 tablet_replica.cc:439] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } }
I20260812 17:48:43.804473 26426 tablet_replica.cc:442] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:43.815065 26421 raft_consensus.cc:482] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:43.815239 26421 raft_consensus.cc:504] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.816781 26421 leader_election.cc:288] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429), aaa101680029404995cd4d34753ff23d (127.13.38.2:32951)
I20260812 17:48:43.821455 26194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.817044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:41429 (local address 127.13.38.3:33489)
0812 17:48:43.817185 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.817203 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.817305 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.818221 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.818224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.818241 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:43.818365 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.818371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.819380 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.819383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.820324 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.820334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.821007 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.821011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.821117 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.821162 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.821240 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.821315 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 17:48:43.822042 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.817787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:33489 (local address 127.13.38.1:41429)
0812 17:48:43.817896 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.817900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.817913 (+ 13us) server_negotiation.cc:407] Connection header received
0812 17:48:43.817979 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.817982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.818026 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.818108 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:43.818492 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.819259 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.820461 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.820867 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.820994 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.821074 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.821820 (+ 746us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.821913 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 17:48:43.822336 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.817475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:54141 (local address 127.13.38.2:32951)
0812 17:48:43.817591 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.817596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.817853 (+ 257us) server_negotiation.cc:407] Connection header received
0812 17:48:43.818057 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.818061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.818114 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.818167 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:48:43.818999 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.819653 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.821191 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.821780 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.821879 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.821939 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.822123 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.822217 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 17:48:43.822458 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.817348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:32951 (local address 127.13.38.3:54141)
0812 17:48:43.817769 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.817782 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.817931 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.818324 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.818327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.818352 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:48:43.818871 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.818879 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.819783 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.819787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.821008 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.821019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.821890 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.821893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.821988 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.822016 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.822183 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.822246 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":5,"thread_start_us":149,"threads_started":1}
I20260812 17:48:43.822449 25857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c115ddab4b49e798906b45d1f9f893" candidate_uuid: "9336693035da4c0b9667233bd92e8cfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7985e770b334cf28cb8a59799b2df2c" is_pre_election: true
I20260812 17:48:43.822600 25857 raft_consensus.cc:2454] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9336693035da4c0b9667233bd92e8cfc in term 0.
I20260812 17:48:43.823024 26182 leader_election.cc:302] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9336693035da4c0b9667233bd92e8cfc, d7985e770b334cf28cb8a59799b2df2c; no voters:
I20260812 17:48:43.823068 26068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c115ddab4b49e798906b45d1f9f893" candidate_uuid: "9336693035da4c0b9667233bd92e8cfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaa101680029404995cd4d34753ff23d" is_pre_election: true
I20260812 17:48:43.823228 26068 raft_consensus.cc:2454] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9336693035da4c0b9667233bd92e8cfc in term 0.
I20260812 17:48:43.823449 26421 raft_consensus.cc:2781] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:43.823542 26421 raft_consensus.cc:482] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:43.823596 26421 raft_consensus.cc:3037] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:43.824915 26421 raft_consensus.cc:504] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.825343 26421 leader_election.cc:288] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [CANDIDATE]: Term 1 election: Requested vote from peers d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429), aaa101680029404995cd4d34753ff23d (127.13.38.2:32951)
I20260812 17:48:43.825874 25857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c115ddab4b49e798906b45d1f9f893" candidate_uuid: "9336693035da4c0b9667233bd92e8cfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7985e770b334cf28cb8a59799b2df2c"
I20260812 17:48:43.826020 25857 raft_consensus.cc:3037] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:43.825990 26068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c115ddab4b49e798906b45d1f9f893" candidate_uuid: "9336693035da4c0b9667233bd92e8cfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaa101680029404995cd4d34753ff23d"
I20260812 17:48:43.826149 26068 raft_consensus.cc:3037] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:43.827196 25857 raft_consensus.cc:2454] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9336693035da4c0b9667233bd92e8cfc in term 1.
I20260812 17:48:43.827363 26068 raft_consensus.cc:2454] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9336693035da4c0b9667233bd92e8cfc in term 1.
I20260812 17:48:43.827623 26182 leader_election.cc:302] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9336693035da4c0b9667233bd92e8cfc, d7985e770b334cf28cb8a59799b2df2c; no voters:
I20260812 17:48:43.827847 26421 raft_consensus.cc:2781] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:43.828109 26421 raft_consensus.cc:686] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [term 1 LEADER]: Becoming Leader. State: Replica: 9336693035da4c0b9667233bd92e8cfc, State: Running, Role: LEADER
I20260812 17:48:43.828413 26421 consensus_queue.cc:237] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:43.830737 26482 tablet_replica.cc:439] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: TxnStatusTablet state changed. Reason: New leader 9336693035da4c0b9667233bd92e8cfc. Latest consensus state: current_term: 1 leader_uuid: "9336693035da4c0b9667233bd92e8cfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } }
I20260812 17:48:43.830859 26482 tablet_replica.cc:442] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: This TxnStatusTablet replica's current role is: LEADER
I20260812 17:48:43.831156 26488 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 17:48:43.831326 26488 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 17:48:43.831229 25614 catalog_manager.cc:5495] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc reported cstate change: term changed from 0 to 1, leader changed from <none> to 9336693035da4c0b9667233bd92e8cfc (127.13.38.3). New cstate: current_term: 1 leader_uuid: "9336693035da4c0b9667233bd92e8cfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } health_report { overall_health: HEALTHY } } }
I20260812 17:48:43.846473 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.841391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:36135 (local address 127.0.0.1:39070)
0812 17:48:43.841654 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.841670 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.841794 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.842171 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.842174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.842186 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:43.842314 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.842321 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.843362 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.843366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.844377 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.844389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.845155 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.845158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.845247 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.845301 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:43.846122 (+ 821us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:43.846142 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.846232 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.846327 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260812 17:48:43.846532 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.841490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39070 (local address 127.13.38.3:36135)
0812 17:48:43.841610 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.841615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.841741 (+ 126us) server_negotiation.cc:407] Connection header received
0812 17:48:43.841956 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.841959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.842025 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.842144 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:43.842437 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.843231 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.844514 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.844853 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.845030 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.845458 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:43.846018 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:43.846133 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.846264 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.846328 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 17:48:43.849929 25857 raft_consensus.cc:1261] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Refusing update from remote peer 9336693035da4c0b9667233bd92e8cfc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:43.849997 26068 raft_consensus.cc:1261] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Refusing update from remote peer 9336693035da4c0b9667233bd92e8cfc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:43.850574 26421 consensus_queue.cc:1035] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:43.850847 26482 consensus_queue.cc:1035] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:43.852526 26426 tablet_replica.cc:439] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c: TxnStatusTablet state changed. Reason: New leader 9336693035da4c0b9667233bd92e8cfc. Latest consensus state: current_term: 1 leader_uuid: "9336693035da4c0b9667233bd92e8cfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } }
I20260812 17:48:43.852674 26426 tablet_replica.cc:442] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:43.852810 26429 tablet_replica.cc:439] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d: TxnStatusTablet state changed. Reason: New leader 9336693035da4c0b9667233bd92e8cfc. Latest consensus state: current_term: 1 leader_uuid: "9336693035da4c0b9667233bd92e8cfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } }
I20260812 17:48:43.852943 26429 tablet_replica.cc:442] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:43.854619 26421 tablet_replica.cc:439] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9336693035da4c0b9667233bd92e8cfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } }
I20260812 17:48:43.854754 26421 tablet_replica.cc:442] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: This TxnStatusTablet replica's current role is: LEADER
I20260812 17:48:43.856633 26482 tablet_replica.cc:439] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9336693035da4c0b9667233bd92e8cfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } }
I20260812 17:48:43.856743 26482 tablet_replica.cc:442] T c1c115ddab4b49e798906b45d1f9f893 P 9336693035da4c0b9667233bd92e8cfc: This TxnStatusTablet replica's current role is: LEADER
I20260812 17:48:43.857954 26433 mvcc.cc:205] Tried to move back new op lower bound from 7317737160083120128 to 7317737160004313088. Current Snapshot: MvccSnapshot[applied={T|T < 7317737160083120128}]
I20260812 17:48:43.858004 26429 tablet_replica.cc:439] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9336693035da4c0b9667233bd92e8cfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } }
I20260812 17:48:43.858119 26429 tablet_replica.cc:442] T c1c115ddab4b49e798906b45d1f9f893 P aaa101680029404995cd4d34753ff23d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:43.859340 26426 tablet_replica.cc:439] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9336693035da4c0b9667233bd92e8cfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } } }
I20260812 17:48:43.859477 26426 tablet_replica.cc:442] T c1c115ddab4b49e798906b45d1f9f893 P d7985e770b334cf28cb8a59799b2df2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 17:48:43.859439 26435 mvcc.cc:205] Tried to move back new op lower bound from 7317737160083120128 to 7317737160004313088. Current Snapshot: MvccSnapshot[applied={T|T < 7317737160083120128}]
I20260812 17:48:43.865376 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.860832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36836 (local address 127.13.38.1:41429)
0812 17:48:43.860981 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.860985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.860998 (+ 13us) server_negotiation.cc:407] Connection header received
0812 17:48:43.861081 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.861084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.861148 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.861264 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:43.861709 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.862545 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.863635 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.864020 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.864129 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.864370 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:43.864982 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:43.865104 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.865153 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.865214 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 17:48:43.876552 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.873079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:45301 (local address 127.13.38.3:36135)
0812 17:48:43.873219 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.873224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.873417 (+ 193us) server_negotiation.cc:407] Connection header received
0812 17:48:43.873581 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.873585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.873638 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.873725 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:43.873994 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.874485 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.875376 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.875659 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.875771 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.876093 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:43.876169 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:43.876249 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.876341 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.876392 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:48:43.876819 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.872892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:36135 (local address 127.13.38.1:45301)
0812 17:48:43.873320 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.873332 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.873460 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.873741 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.873744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.873752 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:43.873875 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.873886 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.874617 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.874621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.875248 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.875257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.875778 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.875781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.875897 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.875942 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:43.876255 (+ 313us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:43.876264 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.876332 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.876410 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":147,"threads_started":1}
I20260812 17:48:43.883976 26068 raft_consensus.cc:1261] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Refusing update from remote peer d7985e770b334cf28cb8a59799b2df2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:43.883976 26299 raft_consensus.cc:1261] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Refusing update from remote peer d7985e770b334cf28cb8a59799b2df2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:43.884498 26426 consensus_queue.cc:1035] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:43.884745 26460 consensus_queue.cc:1035] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:43.908375 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.904008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:35491 (local address 127.13.38.1:41429)
0812 17:48:43.904123 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.904128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.904217 (+ 89us) server_negotiation.cc:407] Connection header received
0812 17:48:43.904328 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.904331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.904381 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.904469 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:43.904694 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.905528 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.907054 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.907405 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.907512 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.907883 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:43.907975 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:43.908050 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:43.908179 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 17:48:43.908233 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 17:48:43.908414 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.903950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:41429 (local address 127.13.38.3:35491)
0812 17:48:43.904122 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:43.904138 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:43.904238 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:43.904473 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:43.904475 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:43.904481 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:43.904593 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.904598 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.905688 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.905695 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.906871 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:43.906886 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.907568 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.907573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:43.907691 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.907744 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:43.908067 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:43.908073 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 17:48:43.908170 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 17:48:43.908254 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 17:48:43.938460 25614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:43.938664 25614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:43.941288 25614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d: Sending DeleteTablet for 3 replicas of tablet 2245ca7dd1434e838889597d11c4e4ca
I20260812 17:48:43.942238 26041 tablet_service.cc:1514] Processing DeleteTablet for tablet 2245ca7dd1434e838889597d11c4e4ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:43 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:54572
I20260812 17:48:43.942413 25823 tablet_service.cc:1514] Processing DeleteTablet for tablet 2245ca7dd1434e838889597d11c4e4ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:43 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:36834
I20260812 17:48:43.942477 26448 tablet_replica.cc:330] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d: stopping tablet replica
I20260812 17:48:43.942600 26443 tablet_replica.cc:330] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c: stopping tablet replica
I20260812 17:48:43.942634 26448 raft_consensus.cc:2229] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:43.942718 26443 raft_consensus.cc:2229] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:43.942785 26448 raft_consensus.cc:2258] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:43.943085 26443 raft_consensus.cc:2258] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:43.943763 26448 ts_tablet_manager.cc:1905] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:43.944415 26264 tablet_service.cc:1514] Processing DeleteTablet for tablet 2245ca7dd1434e838889597d11c4e4ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:43 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:39060
I20260812 17:48:43.944736 26450 tablet_replica.cc:330] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc: stopping tablet replica
I20260812 17:48:43.944880 26450 raft_consensus.cc:2229] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:43.945123 26450 raft_consensus.cc:2258] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:43.947938 26448 ts_tablet_manager.cc:1918] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:43.948096 26448 log.cc:1196] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/2245ca7dd1434e838889597d11c4e4ca
I20260812 17:48:43.948523 26448 ts_tablet_manager.cc:1939] T 2245ca7dd1434e838889597d11c4e4ca P aaa101680029404995cd4d34753ff23d: Deleting consensus metadata
I20260812 17:48:43.949443 25595 catalog_manager.cc:4841] TS aaa101680029404995cd4d34753ff23d (127.13.38.2:32951): tablet 2245ca7dd1434e838889597d11c4e4ca (table test-table [id=2fc60c5b114545c19aa7489ea3dc8fc5]) successfully deleted
I20260812 17:48:43.950153 26443 ts_tablet_manager.cc:1905] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:43.954249 26443 ts_tablet_manager.cc:1918] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:43.954350 26443 log.cc:1196] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/wal/wals/2245ca7dd1434e838889597d11c4e4ca
I20260812 17:48:43.954610 26443 ts_tablet_manager.cc:1939] T 2245ca7dd1434e838889597d11c4e4ca P d7985e770b334cf28cb8a59799b2df2c: Deleting consensus metadata
I20260812 17:48:43.955571 25595 catalog_manager.cc:4841] TS d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429): tablet 2245ca7dd1434e838889597d11c4e4ca (table test-table [id=2fc60c5b114545c19aa7489ea3dc8fc5]) successfully deleted
I20260812 17:48:43.955969 26450 ts_tablet_manager.cc:1905] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:43.958760 26450 ts_tablet_manager.cc:1918] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:43.958850 26450 log.cc:1196] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/2245ca7dd1434e838889597d11c4e4ca
I20260812 17:48:43.959129 26450 ts_tablet_manager.cc:1939] T 2245ca7dd1434e838889597d11c4e4ca P 9336693035da4c0b9667233bd92e8cfc: Deleting consensus metadata
I20260812 17:48:43.960441 25596 catalog_manager.cc:4841] TS 9336693035da4c0b9667233bd92e8cfc (127.13.38.3:36135): tablet 2245ca7dd1434e838889597d11c4e4ca (table test-table [id=2fc60c5b114545c19aa7489ea3dc8fc5]) successfully deleted
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 17:48:43.993088 25696 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:43.984362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43770 (local address 127.13.38.62:42309)
0812 17:48:43.984537 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.984543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.984750 (+ 207us) server_negotiation.cc:407] Connection header received
0812 17:48:43.984900 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.984903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.984963 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.985079 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:43.986034 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.986891 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.988247 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.988598 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.988676 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:43.992945 (+ 4269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43770: BlockingRecv error: recv got EOF from 127.0.0.1:43770 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 17:48:43 dist-test-slave-sb2z krb5kdc[25545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM
I20260812 17:48:44.007215 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:43.995012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43780 (local address 127.13.38.62:42309)
0812 17:48:43.995131 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:43.995135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:43.995366 (+ 231us) server_negotiation.cc:407] Connection header received
0812 17:48:43.995487 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:43.995490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:43.995604 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:43.995702 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:43.996622 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.997395 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.998559 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:43.998932 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:43.999046 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:44.002339 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:44.002343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:44.002346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:44.002377 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:44.004190 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:44.004744 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:44.004749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:44.004751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:44.004873 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:44.006111 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:44.006115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:44.006117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:44.006380 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:44.006503 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:44.006904 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 17:48:44.007043 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 17:48:44.011232 25614 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:48:44.011817 25614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:44.017197 26264 tablet_service.cc:1467] Processing CreateTablet for tablet 726d8533279642ebbfaf05f38b0917f5 (DEFAULT_TABLE table=test-table [id=59e90d49cb1742a3bf88c69be887bf4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:44.017297 26041 tablet_service.cc:1467] Processing CreateTablet for tablet 726d8533279642ebbfaf05f38b0917f5 (DEFAULT_TABLE table=test-table [id=59e90d49cb1742a3bf88c69be887bf4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:44.017524 26264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 726d8533279642ebbfaf05f38b0917f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:44.017635 26041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 726d8533279642ebbfaf05f38b0917f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:44.020229 26416 tablet_bootstrap.cc:492] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d: Bootstrap starting.
I20260812 17:48:44.021164 26416 tablet_bootstrap.cc:654] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:44.021868 25823 tablet_service.cc:1467] Processing CreateTablet for tablet 726d8533279642ebbfaf05f38b0917f5 (DEFAULT_TABLE table=test-table [id=59e90d49cb1742a3bf88c69be887bf4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:44.022159 25823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 726d8533279642ebbfaf05f38b0917f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:44.024838 26420 tablet_bootstrap.cc:492] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c: Bootstrap starting.
I20260812 17:48:44.025823 26420 tablet_bootstrap.cc:654] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:44.027244 26415 tablet_bootstrap.cc:492] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc: Bootstrap starting.
I20260812 17:48:44.028131 26415 tablet_bootstrap.cc:654] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:44.030095 26415 tablet_bootstrap.cc:492] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc: No bootstrap required, opened a new log
I20260812 17:48:44.030179 26415 ts_tablet_manager.cc:1397] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:44.030601 26415 raft_consensus.cc:348] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.030714 26415 raft_consensus.cc:374] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:44.030741 26415 raft_consensus.cc:729] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336693035da4c0b9667233bd92e8cfc, State: Initialized, Role: FOLLOWER
I20260812 17:48:44.030859 26415 consensus_queue.cc:260] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.031109 26415 ts_tablet_manager.cc:1428] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:44.031719 26416 tablet_bootstrap.cc:492] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d: No bootstrap required, opened a new log
I20260812 17:48:44.031837 26416 ts_tablet_manager.cc:1397] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260812 17:48:44.032047 26420 tablet_bootstrap.cc:492] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c: No bootstrap required, opened a new log
I20260812 17:48:44.032130 26420 ts_tablet_manager.cc:1397] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260812 17:48:44.032290 26416 raft_consensus.cc:348] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.032425 26416 raft_consensus.cc:374] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:44.032474 26416 raft_consensus.cc:729] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaa101680029404995cd4d34753ff23d, State: Initialized, Role: FOLLOWER
I20260812 17:48:44.032617 26416 consensus_queue.cc:260] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.032657 26420 raft_consensus.cc:348] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.032756 26420 raft_consensus.cc:374] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:44.032783 26420 raft_consensus.cc:729] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7985e770b334cf28cb8a59799b2df2c, State: Initialized, Role: FOLLOWER
I20260812 17:48:44.032896 26416 ts_tablet_manager.cc:1428] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 17:48:44.032889 26420 consensus_queue.cc:260] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.033205 26420 ts_tablet_manager.cc:1428] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:44.094209 26429 raft_consensus.cc:482] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:44.094398 26429 raft_consensus.cc:504] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.094957 26429 leader_election.cc:288] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429), 9336693035da4c0b9667233bd92e8cfc (127.13.38.3:36135)
I20260812 17:48:44.095480 25857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726d8533279642ebbfaf05f38b0917f5" candidate_uuid: "aaa101680029404995cd4d34753ff23d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7985e770b334cf28cb8a59799b2df2c" is_pre_election: true
I20260812 17:48:44.095511 26299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726d8533279642ebbfaf05f38b0917f5" candidate_uuid: "aaa101680029404995cd4d34753ff23d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336693035da4c0b9667233bd92e8cfc" is_pre_election: true
I20260812 17:48:44.095703 26299 raft_consensus.cc:2454] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaa101680029404995cd4d34753ff23d in term 0.
I20260812 17:48:44.095706 25857 raft_consensus.cc:2454] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaa101680029404995cd4d34753ff23d in term 0.
I20260812 17:48:44.096077 25964 leader_election.cc:302] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9336693035da4c0b9667233bd92e8cfc, aaa101680029404995cd4d34753ff23d; no voters:
I20260812 17:48:44.096290 26429 raft_consensus.cc:2781] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:44.096372 26429 raft_consensus.cc:482] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:44.096438 26429 raft_consensus.cc:3037] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:44.097853 26429 raft_consensus.cc:504] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.098251 26429 leader_election.cc:288] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [CANDIDATE]: Term 1 election: Requested vote from peers d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429), 9336693035da4c0b9667233bd92e8cfc (127.13.38.3:36135)
I20260812 17:48:44.098591 25857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726d8533279642ebbfaf05f38b0917f5" candidate_uuid: "aaa101680029404995cd4d34753ff23d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7985e770b334cf28cb8a59799b2df2c"
I20260812 17:48:44.098716 25857 raft_consensus.cc:3037] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:44.098690 26299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726d8533279642ebbfaf05f38b0917f5" candidate_uuid: "aaa101680029404995cd4d34753ff23d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9336693035da4c0b9667233bd92e8cfc"
I20260812 17:48:44.098816 26299 raft_consensus.cc:3037] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:44.099699 25857 raft_consensus.cc:2454] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaa101680029404995cd4d34753ff23d in term 1.
I20260812 17:48:44.100003 25963 leader_election.cc:302] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaa101680029404995cd4d34753ff23d, d7985e770b334cf28cb8a59799b2df2c; no voters:
I20260812 17:48:44.100206 26429 raft_consensus.cc:2781] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:44.100214 26299 raft_consensus.cc:2454] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaa101680029404995cd4d34753ff23d in term 1.
I20260812 17:48:44.100611 26429 raft_consensus.cc:686] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 1 LEADER]: Becoming Leader. State: Replica: aaa101680029404995cd4d34753ff23d, State: Running, Role: LEADER
I20260812 17:48:44.100793 26429 consensus_queue.cc:237] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.102802 25611 catalog_manager.cc:5495] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d reported cstate change: term changed from 0 to 1, leader changed from <none> to aaa101680029404995cd4d34753ff23d (127.13.38.2). New cstate: current_term: 1 leader_uuid: "aaa101680029404995cd4d34753ff23d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } health_report { overall_health: UNKNOWN } } }
I20260812 17:48:44.125648 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:44.121465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54582 (local address 127.13.38.2:32951)
0812 17:48:44.121983 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:44.121990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:44.122008 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:48:44.122071 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:44.122075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:44.122146 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:44.122233 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:44.122723 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:44.123574 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:44.124441 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:44.124796 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:44.124895 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:44.125180 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:44.125262 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:44.125329 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:44.125438 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 17:48:44.125496 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409}
I20260812 17:48:44.127846 25857 raft_consensus.cc:1261] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Refusing update from remote peer aaa101680029404995cd4d34753ff23d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:44.127942 26299 raft_consensus.cc:1261] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Refusing update from remote peer aaa101680029404995cd4d34753ff23d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:44.128420 26425 consensus_queue.cc:1035] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:44.128603 26429 consensus_queue.cc:1035] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:44.136652 25614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:44.136834 25614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:44.139585 25614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d: Sending DeleteTablet for 3 replicas of tablet 726d8533279642ebbfaf05f38b0917f5
I20260812 17:48:44.140343 26264 tablet_service.cc:1514] Processing DeleteTablet for tablet 726d8533279642ebbfaf05f38b0917f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:44 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:39060
I20260812 17:48:44.140580 26450 tablet_replica.cc:330] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc: stopping tablet replica
I20260812 17:48:44.140736 26450 raft_consensus.cc:2229] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:44.140918 26450 raft_consensus.cc:2258] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:44.141167 25823 tablet_service.cc:1514] Processing DeleteTablet for tablet 726d8533279642ebbfaf05f38b0917f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:44 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:36834
I20260812 17:48:44.141403 26041 tablet_service.cc:1514] Processing DeleteTablet for tablet 726d8533279642ebbfaf05f38b0917f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:44 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:54572
I20260812 17:48:44.141587 26448 tablet_replica.cc:330] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d: stopping tablet replica
I20260812 17:48:44.141713 26448 raft_consensus.cc:2229] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:44.141742 26450 ts_tablet_manager.cc:1905] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:44.141935 26448 raft_consensus.cc:2258] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:44.142223 25613 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:48:44.142655 25613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:48:44.143828 26443 tablet_replica.cc:330] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c: stopping tablet replica
I20260812 17:48:44.144042 26443 raft_consensus.cc:2229] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:44.144227 26443 raft_consensus.cc:2258] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:44.144737 26450 ts_tablet_manager.cc:1918] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:44.144830 26450 log.cc:1196] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/726d8533279642ebbfaf05f38b0917f5
I20260812 17:48:44.144927 26443 ts_tablet_manager.cc:1905] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:44.145043 26448 ts_tablet_manager.cc:1905] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:44.145638 26450 ts_tablet_manager.cc:1939] T 726d8533279642ebbfaf05f38b0917f5 P 9336693035da4c0b9667233bd92e8cfc: Deleting consensus metadata
I20260812 17:48:44.147672 26443 ts_tablet_manager.cc:1918] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:44.147763 26443 log.cc:1196] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-0/wal/wals/726d8533279642ebbfaf05f38b0917f5
I20260812 17:48:44.148008 26448 ts_tablet_manager.cc:1918] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:48:44.148069 26443 ts_tablet_manager.cc:1939] T 726d8533279642ebbfaf05f38b0917f5 P d7985e770b334cf28cb8a59799b2df2c: Deleting consensus metadata
I20260812 17:48:44.148138 26448 log.cc:1196] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/726d8533279642ebbfaf05f38b0917f5
I20260812 17:48:44.148458 26448 ts_tablet_manager.cc:1939] T 726d8533279642ebbfaf05f38b0917f5 P aaa101680029404995cd4d34753ff23d: Deleting consensus metadata
I20260812 17:48:44.148603 25596 catalog_manager.cc:4841] TS 9336693035da4c0b9667233bd92e8cfc (127.13.38.3:36135): tablet 726d8533279642ebbfaf05f38b0917f5 (table test-table [id=59e90d49cb1742a3bf88c69be887bf4f]) successfully deleted
I20260812 17:48:44.148852 25595 catalog_manager.cc:4841] TS d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429): tablet 726d8533279642ebbfaf05f38b0917f5 (table test-table [id=59e90d49cb1742a3bf88c69be887bf4f]) successfully deleted
I20260812 17:48:44.149245 25595 catalog_manager.cc:4841] TS aaa101680029404995cd4d34753ff23d (127.13.38.2:32951): tablet 726d8533279642ebbfaf05f38b0917f5 (table test-table [id=59e90d49cb1742a3bf88c69be887bf4f]) successfully deleted
I20260812 17:48:44.149663 26041 tablet_service.cc:1467] Processing CreateTablet for tablet df8e649553784e86803a6a99c1a6893b (DEFAULT_TABLE table=test-table [id=50e8fc2a729f4d66a9ee3e8b410635a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:44.149991 26041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df8e649553784e86803a6a99c1a6893b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:44.152346 26416 tablet_bootstrap.cc:492] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d: Bootstrap starting.
I20260812 17:48:44.153278 26416 tablet_bootstrap.cc:654] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:44.153929 25823 tablet_service.cc:1467] Processing CreateTablet for tablet df8e649553784e86803a6a99c1a6893b (DEFAULT_TABLE table=test-table [id=50e8fc2a729f4d66a9ee3e8b410635a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:44.154223 25823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df8e649553784e86803a6a99c1a6893b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:44.154388 26416 tablet_bootstrap.cc:492] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d: No bootstrap required, opened a new log
I20260812 17:48:44.154490 26416 ts_tablet_manager.cc:1397] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 17:48:44.154855 26416 raft_consensus.cc:348] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.154980 26416 raft_consensus.cc:374] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:44.155040 26416 raft_consensus.cc:729] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaa101680029404995cd4d34753ff23d, State: Initialized, Role: FOLLOWER
I20260812 17:48:44.155174 26416 consensus_queue.cc:260] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.155479 26416 ts_tablet_manager.cc:1428] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:48:44.156718 26420 tablet_bootstrap.cc:492] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c: Bootstrap starting.
I20260812 17:48:44.156991 26264 tablet_service.cc:1467] Processing CreateTablet for tablet df8e649553784e86803a6a99c1a6893b (DEFAULT_TABLE table=test-table [id=50e8fc2a729f4d66a9ee3e8b410635a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:48:44.157327 26264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df8e649553784e86803a6a99c1a6893b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:44.157691 26420 tablet_bootstrap.cc:654] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:44.158933 26420 tablet_bootstrap.cc:492] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c: No bootstrap required, opened a new log
I20260812 17:48:44.159031 26420 ts_tablet_manager.cc:1397] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 17:48:44.159482 26420 raft_consensus.cc:348] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.159647 26420 raft_consensus.cc:374] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:44.159701 26420 raft_consensus.cc:729] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7985e770b334cf28cb8a59799b2df2c, State: Initialized, Role: FOLLOWER
I20260812 17:48:44.159844 26420 consensus_queue.cc:260] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.159902 26415 tablet_bootstrap.cc:492] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc: Bootstrap starting.
I20260812 17:48:44.160225 26420 ts_tablet_manager.cc:1428] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 17:48:44.160971 26415 tablet_bootstrap.cc:654] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:44.162364 26415 tablet_bootstrap.cc:492] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc: No bootstrap required, opened a new log
I20260812 17:48:44.162475 26415 ts_tablet_manager.cc:1397] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:44.162995 26415 raft_consensus.cc:348] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.163136 26415 raft_consensus.cc:374] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:44.163187 26415 raft_consensus.cc:729] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9336693035da4c0b9667233bd92e8cfc, State: Initialized, Role: FOLLOWER
I20260812 17:48:44.163328 26415 consensus_queue.cc:260] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.163694 26415 ts_tablet_manager.cc:1428] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 17:48:44.231734 26494 raft_consensus.cc:482] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:48:44.231906 26494 raft_consensus.cc:504] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.232342 26494 leader_election.cc:288] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429), aaa101680029404995cd4d34753ff23d (127.13.38.2:32951)
I20260812 17:48:44.232827 25857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df8e649553784e86803a6a99c1a6893b" candidate_uuid: "9336693035da4c0b9667233bd92e8cfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7985e770b334cf28cb8a59799b2df2c" is_pre_election: true
I20260812 17:48:44.232867 26068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df8e649553784e86803a6a99c1a6893b" candidate_uuid: "9336693035da4c0b9667233bd92e8cfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaa101680029404995cd4d34753ff23d" is_pre_election: true
I20260812 17:48:44.232975 25857 raft_consensus.cc:2454] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9336693035da4c0b9667233bd92e8cfc in term 0.
I20260812 17:48:44.233099 26068 raft_consensus.cc:2454] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9336693035da4c0b9667233bd92e8cfc in term 0.
I20260812 17:48:44.233316 26182 leader_election.cc:302] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9336693035da4c0b9667233bd92e8cfc, d7985e770b334cf28cb8a59799b2df2c; no voters:
I20260812 17:48:44.233560 26494 raft_consensus.cc:2781] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:48:44.233642 26494 raft_consensus.cc:482] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:48:44.233696 26494 raft_consensus.cc:3037] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:44.234750 26494 raft_consensus.cc:504] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.235112 26494 leader_election.cc:288] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [CANDIDATE]: Term 1 election: Requested vote from peers d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429), aaa101680029404995cd4d34753ff23d (127.13.38.2:32951)
I20260812 17:48:44.235585 26068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df8e649553784e86803a6a99c1a6893b" candidate_uuid: "9336693035da4c0b9667233bd92e8cfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaa101680029404995cd4d34753ff23d"
I20260812 17:48:44.235597 25857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df8e649553784e86803a6a99c1a6893b" candidate_uuid: "9336693035da4c0b9667233bd92e8cfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7985e770b334cf28cb8a59799b2df2c"
I20260812 17:48:44.235739 26068 raft_consensus.cc:3037] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:44.235754 25857 raft_consensus.cc:3037] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:44.236709 26068 raft_consensus.cc:2454] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9336693035da4c0b9667233bd92e8cfc in term 1.
I20260812 17:48:44.236954 25857 raft_consensus.cc:2454] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9336693035da4c0b9667233bd92e8cfc in term 1.
I20260812 17:48:44.237035 26182 leader_election.cc:302] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9336693035da4c0b9667233bd92e8cfc, aaa101680029404995cd4d34753ff23d; no voters:
I20260812 17:48:44.237227 26494 raft_consensus.cc:2781] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:44.237311 26494 raft_consensus.cc:686] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 1 LEADER]: Becoming Leader. State: Replica: 9336693035da4c0b9667233bd92e8cfc, State: Running, Role: LEADER
I20260812 17:48:44.237483 26494 consensus_queue.cc:237] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } }
I20260812 17:48:44.239305 25614 catalog_manager.cc:5495] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc reported cstate change: term changed from 0 to 1, leader changed from <none> to 9336693035da4c0b9667233bd92e8cfc (127.13.38.3). New cstate: current_term: 1 leader_uuid: "9336693035da4c0b9667233bd92e8cfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9336693035da4c0b9667233bd92e8cfc" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36135 } health_report { overall_health: HEALTHY } } }
I20260812 17:48:44.263355 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:44.259824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.13.38.3:36135)
0812 17:48:44.259935 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:44.259939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:44.259973 (+ 34us) server_negotiation.cc:407] Connection header received
0812 17:48:44.260114 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:44.260117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:44.260166 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:44.260237 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:44.260461 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:44.261276 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:44.262160 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:44.262512 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:44.262608 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:44.262861 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:44.262945 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:44.263011 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:44.263156 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 17:48:44.263214 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 17:48:44.272759 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:44.269451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:43879 (local address 127.13.38.3:36135)
0812 17:48:44.269548 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:44.269552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:44.269566 (+ 14us) server_negotiation.cc:407] Connection header received
0812 17:48:44.269677 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:44.269680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:44.269719 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:44.269786 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:48:44.270072 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:44.270811 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:44.271671 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:44.271997 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:44.272105 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:44.272311 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:48:44.272390 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:48:44.272455 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:44.272536 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 17:48:44.272583 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":32}
I20260812 17:48:44.272758 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:44.269367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:36135 (local address 127.13.38.3:43879)
0812 17:48:44.269490 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:44.269506 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:44.269597 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:44.269818 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:44.269821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:44.269829 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 17:48:44.269924 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:44.269928 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:44.270933 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:44.270935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:44.271505 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:44.271513 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:44.272110 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:44.272113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:44.272175 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:44.272202 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 17:48:44.272474 (+ 272us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 17:48:44.272479 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 17:48:44.272527 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 17:48:44.272612 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":1}
I20260812 17:48:44.278448 26068 raft_consensus.cc:1261] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Refusing update from remote peer 9336693035da4c0b9667233bd92e8cfc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:44.278834 25857 raft_consensus.cc:1261] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Refusing update from remote peer 9336693035da4c0b9667233bd92e8cfc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:48:44.278832 26421 consensus_queue.cc:1035] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaa101680029404995cd4d34753ff23d" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 32951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:44.279337 26421 consensus_queue.cc:1035] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7985e770b334cf28cb8a59799b2df2c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:48:44.313781 25614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:44.313971 25614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:48:44.316252 25614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e89067e90ef43cbad0dfec99f58ac3d: Sending DeleteTablet for 3 replicas of tablet df8e649553784e86803a6a99c1a6893b
I20260812 17:48:44.316998 26041 tablet_service.cc:1514] Processing DeleteTablet for tablet df8e649553784e86803a6a99c1a6893b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:44 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:54572
I20260812 17:48:44.317696 25823 tablet_service.cc:1514] Processing DeleteTablet for tablet df8e649553784e86803a6a99c1a6893b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:44 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:36834
I20260812 17:48:44.317941 26443 tablet_replica.cc:330] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c: stopping tablet replica
I20260812 17:48:44.318025 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 25726
I20260812 17:48:44.318102 26443 raft_consensus.cc:2229] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:44.318245 26443 raft_consensus.cc:2258] T df8e649553784e86803a6a99c1a6893b P d7985e770b334cf28cb8a59799b2df2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:44.330060 26448 tablet_replica.cc:330] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d: stopping tablet replica
I20260812 17:48:44.330242 26448 raft_consensus.cc:2229] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:48:44.330395 26448 raft_consensus.cc:2258] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:44.331183 26448 ts_tablet_manager.cc:1905] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:44.333617 26448 ts_tablet_manager.cc:1918] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:44.333727 26448 log.cc:1196] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/df8e649553784e86803a6a99c1a6893b
I20260812 17:48:44.334071 26448 ts_tablet_manager.cc:1939] T df8e649553784e86803a6a99c1a6893b P aaa101680029404995cd4d34753ff23d: Deleting consensus metadata
I20260812 17:48:44.334860 25595 catalog_manager.cc:4841] TS aaa101680029404995cd4d34753ff23d (127.13.38.2:32951): tablet df8e649553784e86803a6a99c1a6893b (table test-table [id=50e8fc2a729f4d66a9ee3e8b410635a5]) successfully deleted
I20260812 17:48:44.335675 26264 tablet_service.cc:1514] Processing DeleteTablet for tablet df8e649553784e86803a6a99c1a6893b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:48:44 UTC) from {username='oryx', principal='oryx/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:39060
I20260812 17:48:44.335953 26450 tablet_replica.cc:330] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc: stopping tablet replica
I20260812 17:48:44.336117 26450 raft_consensus.cc:2229] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:48:44.336470 26450 raft_consensus.cc:2258] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:48:44.337378 26450 ts_tablet_manager.cc:1905] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:48:44.339473 26450 ts_tablet_manager.cc:1918] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 17:48:44.339608 26450 log.cc:1196] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/df8e649553784e86803a6a99c1a6893b
I20260812 17:48:44.339864 26450 ts_tablet_manager.cc:1939] T df8e649553784e86803a6a99c1a6893b P 9336693035da4c0b9667233bd92e8cfc: Deleting consensus metadata
W20260812 17:48:44.340953 25595 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:41429 (error 108)
I20260812 17:48:44.341017 25596 catalog_manager.cc:4841] TS 9336693035da4c0b9667233bd92e8cfc (127.13.38.3:36135): tablet df8e649553784e86803a6a99c1a6893b (table test-table [id=50e8fc2a729f4d66a9ee3e8b410635a5]) successfully deleted
I20260812 17:48:44.341462 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 25940
I20260812 17:48:44.353744 26557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:44.353247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:41429 (local address 127.0.0.1:36846)
0812 17:48:44.353490 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:44.353588 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:41429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
W20260812 17:48:44.353914 25595 catalog_manager.cc:4584] TS d7985e770b334cf28cb8a59799b2df2c (127.13.38.1:41429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df8e649553784e86803a6a99c1a6893b: Network error: Client connection negotiation failed: client connection to 127.13.38.1:41429: connect: Connection refused (error 111)
I20260812 17:48:44.361541 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 26152
I20260812 17:48:44.375811 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 25565
2026-08-12T17:48:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3437 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:44 dist-test-slave-sb2z krb5kdc[26560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:44 dist-test-slave-sb2z krb5kdc[26560](info): set up 2 sockets
Aug 12 17:48:44 dist-test-slave-sb2z krb5kdc[26560](info): commencing operation
krb5kdc: starting...
W20260812 17:48:45.663148 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:45 dist-test-slave-sb2z krb5kdc[26560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:48:45.812271 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:37121
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:37121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:45.932325 26581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:45.932664 26581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:45.932710 26581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:45.936921 26581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:45.937013 26581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:45.937036 26581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:45.937052 26581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:45.937070 26581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:45.942878 26581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:37121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:37121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:45.944187 26581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:45.945276 26581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:45.954499 26587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:45.954514 26592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:45.954492 26595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:45.955075 26581 server_base.cc:1034] running on GCE node
I20260812 17:48:45.955763 26581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:45.957017 26581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:45.958211 26581 hybrid_clock.cc:648] HybridClock initialized: now 1786556925958195 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T17:48:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1588 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:48:46 dist-test-slave-sb2z krb5kdc[26603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:48:46 dist-test-slave-sb2z krb5kdc[26603](info): set up 2 sockets
Aug 12 17:48:46 dist-test-slave-sb2z krb5kdc[26603](info): commencing operation
krb5kdc: starting...
W20260812 17:48:47.239734 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:48:47 dist-test-slave-sb2z krb5kdc[26603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:48:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:48:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:48:47.379042 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:42375
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:42375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:47.497412 26620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:47.497757 26620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:47.497839 26620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:47.502027 26620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:47.502154 26620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:47.502195 26620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:47.502235 26620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:47.502276 26620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:47.508224 26620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:42375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:42375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:47.509485 26620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:47.510645 26620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:47.519666 26635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:47.519711 26632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:47.519852 26620 server_base.cc:1034] running on GCE node
W20260812 17:48:47.519680 26633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:47.520627 26620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:47.521868 26620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:47.523104 26620 hybrid_clock.cc:648] HybridClock initialized: now 1786556927523082 us; error 34 us; skew 500 ppm
Aug 12 17:48:47 dist-test-slave-sb2z krb5kdc[26603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556927, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:47.526423 26620 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:48:47.527391 26620 webserver.cc:460] Webserver started at http://127.13.38.62:43221/ using document root <none> and password file <none>
I20260812 17:48:47.528092 26620 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:47.528198 26620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:47.528447 26620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:47.530612 26620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b"
format_stamp: "Formatted at 2026-08-12 17:48:47 on dist-test-slave-sb2z"
server_key: "2a761374c2eeac42fdf54d362f67c3e4"
server_key_iv: "75b65b0d90b54258a48c4ce85c86e942"
server_key_version: "encryptionkey@0"
I20260812 17:48:47.531260 26620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b"
format_stamp: "Formatted at 2026-08-12 17:48:47 on dist-test-slave-sb2z"
server_key: "2a761374c2eeac42fdf54d362f67c3e4"
server_key_iv: "75b65b0d90b54258a48c4ce85c86e942"
server_key_version: "encryptionkey@0"
I20260812 17:48:47.535291 26620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 17:48:47.537992 26649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:47.539371 26620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:47.539563 26620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b"
format_stamp: "Formatted at 2026-08-12 17:48:47 on dist-test-slave-sb2z"
server_key: "2a761374c2eeac42fdf54d362f67c3e4"
server_key_iv: "75b65b0d90b54258a48c4ce85c86e942"
server_key_version: "encryptionkey@0"
I20260812 17:48:47.539726 26620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:47.560868 26620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:47.564183 26620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:47.564417 26620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:47.571885 26620 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:42375
I20260812 17:48:47.571902 26724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:42375 every 8 connection(s)
I20260812 17:48:47.572940 26620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:47.575856 26726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:48:47.577210 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 26620
I20260812 17:48:47.577414 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:48:47.577746 13464 external_mini_cluster.cc:1428] Setting key 005c395ee8c48668d7df671c054de9ce
I20260812 17:48:47.583163 26726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b: Bootstrap starting.
I20260812 17:48:47.585974 26726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b: Neither blocks nor log segments found. Creating new log.
I20260812 17:48:47.586934 26726 log.cc:824] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b: Log is configured to *not* fsync() on all Append() calls
Aug 12 17:48:47 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:47.589627 26726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b: No bootstrap required, opened a new log
I20260812 17:48:47.593071 26726 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42375 } }
I20260812 17:48:47.593361 26726 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:48:47.593466 26726 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a2dce499bc4f5fb2c0d6365c4ea77b, State: Initialized, Role: FOLLOWER
I20260812 17:48:47.594038 26726 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42375 } }
I20260812 17:48:47.594211 26726 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:47.594304 26726 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:47.594429 26726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:48:47.595685 26726 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42375 } }
I20260812 17:48:47.595978 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:47.580409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60126 (local address 127.13.38.62:42375)
0812 17:48:47.580852 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:47.580873 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:47.580937 (+ 64us) server_negotiation.cc:407] Connection header received
0812 17:48:47.581927 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:47.581954 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:47.582386 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:47.582771 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:47.583719 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:47.585212 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:47.586263 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:47.586989 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:47.587157 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:47.590285 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:47.590291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:47.590305 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:47.590334 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:47.592781 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:47.593304 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:47.593310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:47.593314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:47.593485 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:47.593786 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:47.593789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:47.593791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:47.594245 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:47.594488 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:47.594770 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 17:48:47.595093 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":130,"threads_started":1}
I20260812 17:48:47.596097 26726 leader_election.cc:302] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6a2dce499bc4f5fb2c0d6365c4ea77b; no voters:
I20260812 17:48:47.596488 26726 leader_election.cc:288] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:48:47.596606 26734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:48:47.596838 26734 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 1 LEADER]: Becoming Leader. State: Replica: f6a2dce499bc4f5fb2c0d6365c4ea77b, State: Running, Role: LEADER
I20260812 17:48:47.598061 26726 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:47.598405 26734 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42375 } }
I20260812 17:48:47.600493 26737 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6a2dce499bc4f5fb2c0d6365c4ea77b. Latest consensus state: current_term: 1 leader_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42375 } } }
I20260812 17:48:47.600520 26736 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42375 } } }
I20260812 17:48:47.600654 26737 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:47.600689 26736 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [sys.catalog]: This master's current role is: LEADER
W20260812 17:48:47.603308 26759 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:48:47.603397 26759 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:48:47.603461 26760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:47.604216 26760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:47.609985 26760 catalog_manager.cc:1347] Generated new cluster ID: 61729a9fd14a4e64959970e4de428c82
I20260812 17:48:47.610086 26760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:47.617714 26760 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:48:47.618822 26760 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:47.627593 26760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b: Generated new TSK 0
I20260812 17:48:47.628479 26760 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:48:47.667389 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:42375
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:47.789095 26766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:47.789404 26766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:47.789453 26766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:47.793793 26766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:47.793879 26766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:47.793942 26766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:47.799731 26766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:42375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:47.800930 26766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:47.802060 26766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:47.812640 26778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:47.812645 26776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:47.812711 26766 server_base.cc:1034] running on GCE node
W20260812 17:48:47.812640 26782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:47.813369 26766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:47.814086 26766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:47.815289 26766 hybrid_clock.cc:648] HybridClock initialized: now 1786556927815265 us; error 41 us; skew 500 ppm
Aug 12 17:48:47 dist-test-slave-sb2z krb5kdc[26603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556927, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:47.818485 26766 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:48:47.819465 26766 webserver.cc:460] Webserver started at http://127.13.38.1:42747/ using document root <none> and password file <none>
I20260812 17:48:47.820106 26766 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:47.820206 26766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:47.820459 26766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:47.822592 26766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "3109d0677d5a477aaaa8ae85f0473f28"
format_stamp: "Formatted at 2026-08-12 17:48:47 on dist-test-slave-sb2z"
server_key: "73208245316e0c03524c2c4bf0bd96f1"
server_key_iv: "4bed0cb9d65417c017519f2592fc09ac"
server_key_version: "encryptionkey@0"
I20260812 17:48:47.823205 26766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "3109d0677d5a477aaaa8ae85f0473f28"
format_stamp: "Formatted at 2026-08-12 17:48:47 on dist-test-slave-sb2z"
server_key: "73208245316e0c03524c2c4bf0bd96f1"
server_key_iv: "4bed0cb9d65417c017519f2592fc09ac"
server_key_version: "encryptionkey@0"
I20260812 17:48:47.827292 26766 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 17:48:47.830032 26791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:47.831346 26766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:47.831498 26766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "3109d0677d5a477aaaa8ae85f0473f28"
format_stamp: "Formatted at 2026-08-12 17:48:47 on dist-test-slave-sb2z"
server_key: "73208245316e0c03524c2c4bf0bd96f1"
server_key_iv: "4bed0cb9d65417c017519f2592fc09ac"
server_key_version: "encryptionkey@0"
I20260812 17:48:47.831687 26766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:47.847672 26766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:47.850708 26766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:47.850925 26766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:47.851581 26766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:47.852459 26766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:47.852533 26766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:47.852638 26766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:47.852692 26766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:47.862334 26766 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:35869
I20260812 17:48:47.862363 26978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:35869 every 8 connection(s)
I20260812 17:48:47.863298 26766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:47.865808 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 26766
I20260812 17:48:47.865953 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:48:47.866309 13464 external_mini_cluster.cc:1428] Setting key 590aa86f1b44262978660661da97bcdb
Aug 12 17:48:47 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556927, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:48:47.879316 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:47.865222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:60723 (local address 127.13.38.62:42375)
0812 17:48:47.865403 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:47.865409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:47.866465 (+ 1056us) server_negotiation.cc:407] Connection header received
0812 17:48:47.868153 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:47.868156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:47.868198 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:47.868270 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:47.869970 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:47.870855 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:47.872272 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:47.872530 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:47.872656 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:47.875614 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:47.875617 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:47.875619 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:47.875644 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:47.877108 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:47.877733 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:47.877737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:47.877739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:47.877826 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:47.878203 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:47.878207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:47.878208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:47.878411 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:47.878536 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:47.879065 (+ 529us) server_negotiation.cc:299] Negotiation successful
0812 17:48:47.879183 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
I20260812 17:48:47.880398 26982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:47.865651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:60723)
0812 17:48:47.866285 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:47.866349 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:47.867925 (+ 1576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:47.868463 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:47.868474 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:47.869032 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:47.869772 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:47.869796 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:47.871012 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:47.871016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:47.872135 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:47.872147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:47.872705 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:47.872710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:47.872881 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:47.873569 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:47.873586 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:47.875386 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:47.877245 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:47.877251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:47.877269 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:47.877621 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:47.877931 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:47.877934 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:47.877936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:47.878100 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:47.878562 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:47.878568 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:47.878831 (+ 263us) client_negotiation.cc:769] Sending connection context
0812 17:48:47.879012 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 17:48:47.879289 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":128,"threads_started":1}
I20260812 17:48:47.882016 26979 heartbeater.cc:347] Connected to a master server at 127.13.38.62:42375
I20260812 17:48:47.882337 26979 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:47.882947 26979 heartbeater.cc:511] Master 127.13.38.62:42375 requested a full tablet report, sending...
I20260812 17:48:47.884699 26670 ts_manager.cc:194] Registered new tserver with Master: 3109d0677d5a477aaaa8ae85f0473f28 (127.13.38.1:35869)
I20260812 17:48:47.885960 26670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:60723
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:48:47.899322 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:42375
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:48.020358 26985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:48.020673 26985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:48.020720 26985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:48.025087 26985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:48.025164 26985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:48.025260 26985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:48.030933 26985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:42375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:48.032218 26985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:48.033322 26985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:48.044070 26993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:48.044101 26994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:48.044245 26985 server_base.cc:1034] running on GCE node
W20260812 17:48:48.044087 26996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:48.044845 26985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:48.045490 26985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:48.046663 26985 hybrid_clock.cc:648] HybridClock initialized: now 1786556928046637 us; error 36 us; skew 500 ppm
Aug 12 17:48:48 dist-test-slave-sb2z krb5kdc[26603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556928, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:48.053529 26985 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:48:48.054574 26985 webserver.cc:460] Webserver started at http://127.13.38.2:43971/ using document root <none> and password file <none>
I20260812 17:48:48.055189 26985 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:48.055258 26985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:48.055452 26985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:48.057760 26985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "6ad3745042554c2f940cacb4702f4474"
format_stamp: "Formatted at 2026-08-12 17:48:48 on dist-test-slave-sb2z"
server_key: "1287f4302148a50dbfc5ae7387e0617e"
server_key_iv: "fcdfb6fa563c14f432ea8ffed4a3d21d"
server_key_version: "encryptionkey@0"
I20260812 17:48:48.058343 26985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "6ad3745042554c2f940cacb4702f4474"
format_stamp: "Formatted at 2026-08-12 17:48:48 on dist-test-slave-sb2z"
server_key: "1287f4302148a50dbfc5ae7387e0617e"
server_key_iv: "fcdfb6fa563c14f432ea8ffed4a3d21d"
server_key_version: "encryptionkey@0"
I20260812 17:48:48.062587 26985 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:48:48.065374 27010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:48.066747 26985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:48.066895 26985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "6ad3745042554c2f940cacb4702f4474"
format_stamp: "Formatted at 2026-08-12 17:48:48 on dist-test-slave-sb2z"
server_key: "1287f4302148a50dbfc5ae7387e0617e"
server_key_iv: "fcdfb6fa563c14f432ea8ffed4a3d21d"
server_key_version: "encryptionkey@0"
I20260812 17:48:48.067046 26985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:48.087082 26985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:48.090677 26985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:48.090849 26985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:48.091428 26985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:48.092334 26985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:48.092384 26985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:48.092492 26985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:48.092526 26985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:48.102473 26985 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:39457
I20260812 17:48:48.102491 27192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:39457 every 8 connection(s)
I20260812 17:48:48.103386 26985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:48.108181 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 26985
I20260812 17:48:48.108333 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:48:48.108631 13464 external_mini_cluster.cc:1428] Setting key 38adde1a0b628f2795ef8459adca4b54
Aug 12 17:48:48 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556928, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:48.118360 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:48.105331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:57513 (local address 127.13.38.62:42375)
0812 17:48:48.105557 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:48.105562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:48.106793 (+ 1231us) server_negotiation.cc:407] Connection header received
0812 17:48:48.107818 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:48.107821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:48.107907 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:48.108017 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:48.109384 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.109928 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.111140 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.111367 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.111492 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:48.114392 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:48.114396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:48.114399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:48.114428 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:48.116065 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:48.116658 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:48.116662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:48.116664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:48.116757 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:48.117126 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:48.117130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:48.117132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:48.117336 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:48.117448 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:48.118036 (+ 588us) server_negotiation.cc:299] Negotiation successful
0812 17:48:48.118182 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 17:48:48.119185 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:48.105848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:57513)
0812 17:48:48.106635 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:48.106678 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:48.107598 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:48.108207 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:48.108215 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:48.108707 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:48.109230 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:48.109243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.110087 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.110090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:48.111001 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:48.111010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.111555 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.111560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:48.111742 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:48.112424 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:48.112442 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:48.114205 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:48.116204 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:48.116211 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:48.116229 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:48.116526 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:48.116858 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:48.116861 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:48.116863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:48.117019 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:48.117448 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:48.117456 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:48.117733 (+ 277us) client_negotiation.cc:769] Sending connection context
0812 17:48:48.117933 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 17:48:48.118299 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:48:48.120676 27193 heartbeater.cc:347] Connected to a master server at 127.13.38.62:42375
I20260812 17:48:48.120935 27193 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:48.121560 27193 heartbeater.cc:511] Master 127.13.38.62:42375 requested a full tablet report, sending...
I20260812 17:48:48.122687 26670 ts_manager.cc:194] Registered new tserver with Master: 6ad3745042554c2f940cacb4702f4474 (127.13.38.2:39457)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
I20260812 17:48:48.123385 26670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:57513
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:48:48.143848 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:42375
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:48.265863 27200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:48.266259 27200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:48.266342 27200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:48.270646 27200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:48.270751 27200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:48.270876 27200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:48.276654 27200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:42375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:48.277829 27200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:48.278903 27200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:48.289690 27210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:48.289685 27207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:48.289685 27212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:48.290149 27200 server_base.cc:1034] running on GCE node
I20260812 17:48:48.290640 27200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:48.291384 27200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:48.292567 27200 hybrid_clock.cc:648] HybridClock initialized: now 1786556928292548 us; error 34 us; skew 500 ppm
Aug 12 17:48:48 dist-test-slave-sb2z krb5kdc[26603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556928, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:48.295465 27200 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:48:48.296429 27200 webserver.cc:460] Webserver started at http://127.13.38.3:45143/ using document root <none> and password file <none>
I20260812 17:48:48.297031 27200 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:48.297127 27200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:48.297374 27200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:48:48.299821 27200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "d037fa7b08fb4a9b8dd7de3d5959fa65"
format_stamp: "Formatted at 2026-08-12 17:48:48 on dist-test-slave-sb2z"
server_key: "c02d5314deadf55a5d8c9acf819dc4e6"
server_key_iv: "15b477ba2831be0ff1940a6a908ad577"
server_key_version: "encryptionkey@0"
I20260812 17:48:48.300441 27200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "d037fa7b08fb4a9b8dd7de3d5959fa65"
format_stamp: "Formatted at 2026-08-12 17:48:48 on dist-test-slave-sb2z"
server_key: "c02d5314deadf55a5d8c9acf819dc4e6"
server_key_iv: "15b477ba2831be0ff1940a6a908ad577"
server_key_version: "encryptionkey@0"
I20260812 17:48:48.304544 27200 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:48:48.307380 27224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:48.308863 27200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:48:48.309067 27200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "d037fa7b08fb4a9b8dd7de3d5959fa65"
format_stamp: "Formatted at 2026-08-12 17:48:48 on dist-test-slave-sb2z"
server_key: "c02d5314deadf55a5d8c9acf819dc4e6"
server_key_iv: "15b477ba2831be0ff1940a6a908ad577"
server_key_version: "encryptionkey@0"
I20260812 17:48:48.309216 27200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:48.326010 27200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:48.329442 27200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:48.329715 27200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:48.330356 27200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:48.331254 27200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:48.331305 27200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:48.331349 27200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:48.331373 27200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:48.341562 27200 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:42683
I20260812 17:48:48.341584 27403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:42683 every 8 connection(s)
I20260812 17:48:48.342621 27200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:48.352447 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 27200
Aug 12 17:48:48 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556928, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:48:48.352612 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:48:48.352902 13464 external_mini_cluster.cc:1428] Setting key ea07793ef487df7077a6b0e5abb7eecc
I20260812 17:48:48.357175 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:48.344856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:53105 (local address 127.13.38.62:42375)
0812 17:48:48.345091 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:48.345095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:48.346106 (+ 1011us) server_negotiation.cc:407] Connection header received
0812 17:48:48.347027 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:48.347030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:48.347092 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:48.347167 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:48.348539 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.349078 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.350270 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.350479 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.350557 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:48.353232 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:48.353238 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:48.353241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:48.353274 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:48.354908 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:48.355559 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:48.355562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:48.355564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:48.355655 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:48.356022 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:48.356026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:48.356028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:48.356235 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:48.356347 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:48.356867 (+ 520us) server_negotiation.cc:299] Negotiation successful
0812 17:48:48.356998 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260812 17:48:48.358036 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:48.345197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:53105)
0812 17:48:48.345912 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:48.345968 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:48.346829 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:48.347348 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:48.347355 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:48.347857 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:48.348381 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:48.348396 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.349229 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.349232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:48.350150 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:48.350160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.350623 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.350626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:48.350779 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:48.351338 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:48.351355 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:48.353041 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:48:48.355045 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:48.355052 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:48.355075 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:48.355420 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:48.355755 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:48:48.355758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:48:48.355760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:48:48.355910 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:48:48.356347 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:48:48.356355 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:48:48.356622 (+ 267us) client_negotiation.cc:769] Sending connection context
0812 17:48:48.356824 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 17:48:48.357186 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":118,"threads_started":1}
I20260812 17:48:48.359155 27405 heartbeater.cc:347] Connected to a master server at 127.13.38.62:42375
I20260812 17:48:48.359424 27405 heartbeater.cc:464] Registering TS with master...
I20260812 17:48:48.359973 27405 heartbeater.cc:511] Master 127.13.38.62:42375 requested a full tablet report, sending...
I20260812 17:48:48.361076 26670 ts_manager.cc:194] Registered new tserver with Master: d037fa7b08fb4a9b8dd7de3d5959fa65 (127.13.38.3:42683)
I20260812 17:48:48.361676 26670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:53105
I20260812 17:48:48.367233 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab.
Entry for principal oryx/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab.
WARNING: no policy specified for oryx/127.13.38.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab.
Entry for principal oryx/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab.
WARNING: no policy specified for oryx/127.13.38.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.38.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab.
Entry for principal oryx/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab.
I20260812 17:48:48.460084 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 26766
I20260812 17:48:48.473012 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 26985
I20260812 17:48:48.489461 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 27200
I20260812 17:48:48.502698 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 26620
I20260812 17:48:48.517648 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:42375
--webserver_interface=127.13.38.62
--webserver_port=43221
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:42375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:48:48.642537 27421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:48.642832 27421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:48.642874 27421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:48.646996 27421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:48:48.647068 27421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:48.647089 27421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:48.647106 27421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:48:48.647121 27421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:48:48.652611 27421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:42375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:42375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=43221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:48.653687 27421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:48.654722 27421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:48.663880 27427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:48.663880 27431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:48.663887 27428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:48.664039 27421 server_base.cc:1034] running on GCE node
I20260812 17:48:48.664826 27421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:48.666007 27421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:48.667209 27421 hybrid_clock.cc:648] HybridClock initialized: now 1786556928667192 us; error 29 us; skew 500 ppm
Aug 12 17:48:48 dist-test-slave-sb2z krb5kdc[26603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556928, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:48.670245 27421 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:48:48.671187 27421 webserver.cc:460] Webserver started at http://127.13.38.62:43221/ using document root <none> and password file <none>
I20260812 17:48:48.671814 27421 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:48.671898 27421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:48.676062 27421 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:48:48.678367 27443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:48.679762 27421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:48.679874 27421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b"
format_stamp: "Formatted at 2026-08-12 17:48:47 on dist-test-slave-sb2z"
server_key: "2a761374c2eeac42fdf54d362f67c3e4"
server_key_iv: "75b65b0d90b54258a48c4ce85c86e942"
server_key_version: "encryptionkey@0"
I20260812 17:48:48.680439 27421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:48.694361 27421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:48.697789 27421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:48.697960 27421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:48.705747 27421 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:42375
I20260812 17:48:48.705843 27536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:42375 every 8 connection(s)
I20260812 17:48:48.706899 27421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:48:48.711568 27537 sys_catalog.cc:263] Verifying existing consensus state
I20260812 17:48:48.713338 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b: Bootstrap starting.
I20260812 17:48:48.715754 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 27421
I20260812 17:48:48.716301 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:35869
--local_ip_for_outbound_sockets=127.13.38.1
--tserver_master_addrs=127.13.38.62:42375
--webserver_port=42747
--webserver_interface=127.13.38.1
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.38.1
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:48:48.732120 27537 log.cc:824] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b: Log is configured to *not* fsync() on all Append() calls
I20260812 17:48:48.739467 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 17:48:48.739989 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b: Bootstrap complete.
I20260812 17:48:48.744809 27537 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42375 } }
I20260812 17:48:48.745311 27537 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a2dce499bc4f5fb2c0d6365c4ea77b, State: Initialized, Role: FOLLOWER
I20260812 17:48:48.745893 27537 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42375 } }
I20260812 17:48:48.746023 27537 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:48:48.746119 27537 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:48:48.746227 27537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 1 FOLLOWER]: Advancing to term 2
I20260812 17:48:48.748543 27537 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42375 } }
I20260812 17:48:48.748939 27537 leader_election.cc:302] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6a2dce499bc4f5fb2c0d6365c4ea77b; no voters:
I20260812 17:48:48.749341 27537 leader_election.cc:288] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 17:48:48.749444 27543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 2 FOLLOWER]: Leader election won for term 2
I20260812 17:48:48.749725 27543 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [term 2 LEADER]: Becoming Leader. State: Replica: f6a2dce499bc4f5fb2c0d6365c4ea77b, State: Running, Role: LEADER
I20260812 17:48:48.750142 27543 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42375 } }
I20260812 17:48:48.750975 27537 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:48:48.751889 27544 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42375 } } }
I20260812 17:48:48.752070 27544 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:48.753000 27558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:48:48.753300 27546 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6a2dce499bc4f5fb2c0d6365c4ea77b. Latest consensus state: current_term: 2 leader_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a2dce499bc4f5fb2c0d6365c4ea77b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42375 } } }
I20260812 17:48:48.753417 27546 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b [sys.catalog]: This master's current role is: LEADER
I20260812 17:48:48.756695 27558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:48:48.757452 27558 catalog_manager.cc:1259] Loaded cluster ID: 61729a9fd14a4e64959970e4de428c82
I20260812 17:48:48.757550 27558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:48:48.760082 27558 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:48:48.760911 27558 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f6a2dce499bc4f5fb2c0d6365c4ea77b: Loaded TSK: 0
I20260812 17:48:48.762261 27558 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 17:48:48.844003 27539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:48.844319 27539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:48.844412 27539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:48.848511 27539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:48.848574 27539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:48.848660 27539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:48:48.854350 27539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.1.keytab
--principal=oryx/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:35869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=42747
--tserver_master_addrs=127.13.38.62:42375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:48.855471 27539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:48.856626 27539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:48.867137 27577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:48.867133 27579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:48.867287 27539 server_base.cc:1034] running on GCE node
W20260812 17:48:48.867172 27576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:48.867960 27539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:48.868695 27539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:48.869884 27539 hybrid_clock.cc:648] HybridClock initialized: now 1786556928869850 us; error 52 us; skew 500 ppm
Aug 12 17:48:48 dist-test-slave-sb2z krb5kdc[26603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556928, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:48.873024 27539 init.cc:377] Logged in from keytab as oryx/127.13.38.1@KRBTEST.COM (short username oryx)
I20260812 17:48:48.874078 27539 webserver.cc:460] Webserver started at http://127.13.38.1:42747/ using document root <none> and password file <none>
I20260812 17:48:48.874711 27539 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:48.874792 27539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:48.879174 27539 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:48:48.881669 27587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:48.883224 27539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 17:48:48.883404 27539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "3109d0677d5a477aaaa8ae85f0473f28"
format_stamp: "Formatted at 2026-08-12 17:48:47 on dist-test-slave-sb2z"
server_key: "73208245316e0c03524c2c4bf0bd96f1"
server_key_iv: "4bed0cb9d65417c017519f2592fc09ac"
server_key_version: "encryptionkey@0"
I20260812 17:48:48.884232 27539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:48.895494 27539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:48.898928 27539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:48.899163 27539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:48.899906 27539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:48.900810 27539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:48.900899 27539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:48.900992 27539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:48.901044 27539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:48.910933 27539 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:35869
I20260812 17:48:48.911126 27772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:35869 every 8 connection(s)
I20260812 17:48:48.912153 27539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:48:48.915889 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 27539
I20260812 17:48:48.916366 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:39457
--local_ip_for_outbound_sockets=127.13.38.2
--tserver_master_addrs=127.13.38.62:42375
--webserver_port=43971
--webserver_interface=127.13.38.2
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.38.2
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 17:48:48 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:48 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 17:48:48.926672 27776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:48.914707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:56499 (local address 127.13.38.62:42375)
0812 17:48:48.915315 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:48.915344 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:48.915901 (+ 557us) server_negotiation.cc:407] Connection header received
0812 17:48:48.917167 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:48.917178 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:48.917469 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:48.917798 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:48.918997 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.920066 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.921243 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.921726 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.921904 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:48.925890 (+ 3986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:56499: BlockingRecv error: recv got EOF from 127.13.38.1:56499 (error 108)
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":148,"threads_started":1}
I20260812 17:48:48.926692 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:48.914820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:56499)
0812 17:48:48.915723 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:48.915771 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:48.916740 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:48.917878 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:48.917886 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:48.918347 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:48.918807 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:48.918821 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.920209 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.920212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:48.921101 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:48.921110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.921908 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.921913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:48.922116 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:48.923041 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:48.923058 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:48.925652 (+ 2594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":165,"threads_started":1}
W20260812 17:48:48.927255 27773 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:42375 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.38.62:42375: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Aug 12 17:48:48 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:48 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:48.933920 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:48.927849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:46279)
0812 17:48:48.928092 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:48.928110 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:48.928206 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:48.928541 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:48.928547 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:48.928756 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:48.928880 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:48.928899 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.929664 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.929667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:48.930569 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:48.930579 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.931031 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.931034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:48.931130 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:48.931694 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:48.931709 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:48.933762 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146}
W20260812 17:48:48.934017 27776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:48.928060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:46279 (local address 127.13.38.62:42375)
0812 17:48:48.928182 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:48.928186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:48.928200 (+ 14us) server_negotiation.cc:407] Connection header received
0812 17:48:48.928350 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:48.928352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:48.928401 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:48.928489 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:48.929037 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.929554 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.930698 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.930910 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.931005 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:48.933895 (+ 2890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:46279: BlockingRecv error: recv got EOF from 127.13.38.1:46279 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 17:48:48 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:48 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:48.940135 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:48.934629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:37029)
0812 17:48:48.934749 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:48.934765 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:48.934838 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:48.935146 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:48.935148 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:48.935276 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:48.935347 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:48.935351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.936410 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.936413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:48.936925 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:48.936932 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.937570 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.937573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:48.937643 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:48.938155 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:48.938167 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:48.939981 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 17:48:48.940420 27773 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 17:48:48.940588 27776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:48.934728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:37029 (local address 127.13.38.62:42375)
0812 17:48:48.934872 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:48.934878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:48.934910 (+ 32us) server_negotiation.cc:407] Connection header received
0812 17:48:48.934956 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:48.934961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:48.935015 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:48.935112 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:48.935451 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.936285 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.937036 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:48.937445 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:48.937553 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:48.940119 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:37029: BlockingRecv error: recv got EOF from 127.13.38.1:37029 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 17:48:49.043769 27780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:49.044076 27780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:49.044122 27780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:49.048252 27780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:49.048326 27780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:49.048429 27780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:48:49.054191 27780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.2.keytab
--principal=oryx/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:39457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=43971
--tserver_master_addrs=127.13.38.62:42375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:49.055406 27780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:49.056885 27780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 17:48:49.067787 27780 server_base.cc:1034] running on GCE node
W20260812 17:48:49.067811 27787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:49.067780 27788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:49.067799 27791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:49.068521 27780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:49.069325 27780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:49.070573 27780 hybrid_clock.cc:648] HybridClock initialized: now 1786556929070540 us; error 39 us; skew 500 ppm
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556929, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:49.073624 27780 init.cc:377] Logged in from keytab as oryx/127.13.38.2@KRBTEST.COM (short username oryx)
I20260812 17:48:49.074704 27780 webserver.cc:460] Webserver started at http://127.13.38.2:43971/ using document root <none> and password file <none>
I20260812 17:48:49.075320 27780 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:49.075446 27780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:49.079741 27780 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 17:48:49.082052 27799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:49.083388 27780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:48:49.083581 27780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "6ad3745042554c2f940cacb4702f4474"
format_stamp: "Formatted at 2026-08-12 17:48:48 on dist-test-slave-sb2z"
server_key: "1287f4302148a50dbfc5ae7387e0617e"
server_key_iv: "fcdfb6fa563c14f432ea8ffed4a3d21d"
server_key_version: "encryptionkey@0"
I20260812 17:48:49.084256 27780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:49.097376 27780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:49.100795 27780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:49.101044 27780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:49.101702 27780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:49.102597 27780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:49.102674 27780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:49.102751 27780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:49.102797 27780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:49.112857 27780 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:39457
I20260812 17:48:49.112886 27987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:39457 every 8 connection(s)
I20260812 17:48:49.113960 27780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:48:49.114744 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 27780
I20260812 17:48:49.115234 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:42683
--local_ip_for_outbound_sockets=127.13.38.3
--tserver_master_addrs=127.13.38.62:42375
--webserver_port=45143
--webserver_interface=127.13.38.3
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:43225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.38.3
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 17:48:49.128756 27776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:49.116114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:44987 (local address 127.13.38.62:42375)
0812 17:48:49.116289 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:49.116293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:49.117645 (+ 1352us) server_negotiation.cc:407] Connection header received
0812 17:48:49.119029 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:49.119032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:49.119085 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:49.119166 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:49.121088 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.121728 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.123756 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.123995 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.124082 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.128644 (+ 4562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:44987: BlockingRecv error: recv got EOF from 127.13.38.2:44987 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 17:48:49.129842 27991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:49.116568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:44987)
0812 17:48:49.117440 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:49.117506 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:49.118756 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:49.119399 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:49.119412 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:49.120009 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:49.120800 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.120827 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.121894 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.121902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.123590 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.123606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.124191 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.124198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.124468 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.125193 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:49.125223 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:49.128482 (+ 3259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":113,"threads_started":1}
W20260812 17:48:49.130396 27989 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:42375 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.38.62:42375: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:49.136387 27991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:49.130909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:34647)
0812 17:48:49.131025 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:49.131042 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:49.131131 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:49.131438 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:49.131441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:49.131644 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:49.131751 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.131758 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.132521 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.132528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.133100 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.133109 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.133590 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.133593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.133695 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.134237 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:49.134250 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:49.136237 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 17:48:49.136629 27776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:49.131004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:34647 (local address 127.13.38.62:42375)
0812 17:48:49.131138 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:49.131142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:49.131159 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:48:49.131229 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:49.131232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:49.131282 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:49.131399 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:49.131878 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.132402 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.133233 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.133462 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.133583 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.136483 (+ 2900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:34647: BlockingRecv error: recv got EOF from 127.13.38.2:34647 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:49.142588 27991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:49.137333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:40219)
0812 17:48:49.137440 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:49.137456 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:49.137539 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:49.137767 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:49.137770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:49.137932 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:49.138013 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.138019 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.138752 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.138757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.139294 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.139302 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.139872 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.139876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.139957 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.140495 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:49.140508 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:49.142425 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 17:48:49.142694 27776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:49.137430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:40219 (local address 127.13.38.62:42375)
0812 17:48:49.137549 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:49.137553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:49.137569 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:48:49.137613 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:49.137616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:49.137657 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:49.137750 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:49.138129 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.138639 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.139430 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.139749 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.139856 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.142564 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:40219: BlockingRecv error: recv got EOF from 127.13.38.2:40219 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 17:48:49.142989 27989 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 17:48:49.243326 27992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:48:49.243727 27992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:48:49.243817 27992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:48:49.247972 27992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:48:49.248050 27992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:48:49.248119 27992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:48:49.254405 27992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:43225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/krb5kdc/oryx_127.13.38.3.keytab
--principal=oryx/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:42683
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=45143
--tserver_master_addrs=127.13.38.62:42375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:48:49.255621 27992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:48:49.256798 27992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:48:49.267321 27998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:49.267339 27992 server_base.cc:1034] running on GCE node
W20260812 17:48:49.267318 27999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:48:49.267375 28006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:48:49.268044 27992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:48:49.268743 27992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:48:49.269938 27992 hybrid_clock.cc:648] HybridClock initialized: now 1786556929269895 us; error 57 us; skew 500 ppm
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556929, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:48:49.273083 27992 init.cc:377] Logged in from keytab as oryx/127.13.38.3@KRBTEST.COM (short username oryx)
I20260812 17:48:49.274248 27992 webserver.cc:460] Webserver started at http://127.13.38.3:45143/ using document root <none> and password file <none>
I20260812 17:48:49.274983 27992 fs_manager.cc:359] Metadata directory not provided
I20260812 17:48:49.275123 27992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:48:49.279708 27992 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 17:48:49.282162 28019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:49.283761 27992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:48:49.283936 27992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "d037fa7b08fb4a9b8dd7de3d5959fa65"
format_stamp: "Formatted at 2026-08-12 17:48:48 on dist-test-slave-sb2z"
server_key: "c02d5314deadf55a5d8c9acf819dc4e6"
server_key_iv: "15b477ba2831be0ff1940a6a908ad577"
server_key_version: "encryptionkey@0"
I20260812 17:48:49.284755 27992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:48:49.296743 27992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:48:49.299968 27992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:48:49.300346 27992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:48:49.301210 27992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:48:49.302165 27992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:48:49.302244 27992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:49.302318 27992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:48:49.302363 27992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:48:49.313120 27992 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:42683
I20260812 17:48:49.313148 28191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:42683 every 8 connection(s)
I20260812 17:48:49.314304 27992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:48:49.314961 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 27992
I20260812 17:48:49.327723 27776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:49.315873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60132 (local address 127.13.38.62:42375)
0812 17:48:49.316034 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:49.316041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:49.316263 (+ 222us) server_negotiation.cc:407] Connection header received
0812 17:48:49.316447 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:49.316450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:49.316509 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:49.316590 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:49.317678 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.319017 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.320325 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.320737 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.320857 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.322654 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:48:49.322661 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:48:49.322685 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:48:49.322720 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:48:49.324733 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:49.325677 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:49.325684 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:49.325690 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:49.325838 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:48:49.326167 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:48:49.326171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:48:49.326172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:48:49.326663 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:48:49.326874 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 17:48:49.327210 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 17:48:49.327481 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 17:48:49.330794 28207 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:49.317943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:46677 (local address 127.13.38.62:42375)
0812 17:48:49.318201 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:49.318204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:49.318738 (+ 534us) server_negotiation.cc:407] Connection header received
0812 17:48:49.319935 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:49.319938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:49.320061 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:49.320172 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:49.322433 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.323589 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.325422 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.325809 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.325929 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.330647 (+ 4718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:46677: BlockingRecv error: recv got EOF from 127.13.38.3:46677 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":84,"threads_started":1}
I20260812 17:48:49.331574 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:49.317089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:46677)
0812 17:48:49.318581 (+ 1492us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:49.318624 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:49.319666 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:49.320513 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:49.320521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:49.321138 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:49.322237 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.322254 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.323770 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.323774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.325261 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.325273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.326247 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.326252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.326459 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.327695 (+ 1236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:49.327717 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:49.330533 (+ 2816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1285,"thread_start_us":163,"threads_started":1}
W20260812 17:48:49.332123 28192 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:42375 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.38.62:42375: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:49.337697 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:49.332554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:42341)
0812 17:48:49.332752 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:49.332768 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:49.332856 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:49.333167 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:49.333169 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:49.333333 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:49.333430 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.333435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.334176 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.334180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.334740 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.334749 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.335225 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.335228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.335324 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.335842 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:49.335853 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:49.337598 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 17:48:49.337878 28207 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:49.332649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:42341 (local address 127.13.38.62:42375)
0812 17:48:49.332796 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:49.332802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:49.332818 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:48:49.332963 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:49.332966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:49.333016 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:49.333137 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:49.333548 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.334054 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.334873 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.335100 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.335210 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.337723 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:42341: BlockingRecv error: recv got EOF from 127.13.38.3:42341 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:49.343379 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:49.338291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:43835)
0812 17:48:49.338425 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:49.338442 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:49.338523 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:49.338783 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:49.338785 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:49.338914 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:49.338993 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.338998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.339732 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.339735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.340320 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.340329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.340763 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.340766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.340831 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.341299 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:49.341311 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:49.343285 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 17:48:49.343600 28207 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:49.338446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:43835 (local address 127.13.38.62:42375)
0812 17:48:49.338604 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:49.338607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:49.338619 (+ 12us) server_negotiation.cc:407] Connection header received
0812 17:48:49.338649 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:49.338650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:49.338685 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:49.338750 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:49.339090 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.339625 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.340437 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.340651 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.340729 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.343392 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:43835: BlockingRecv error: recv got EOF from 127.13.38.3:43835 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 17:48:49.343634 28192 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:49 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:49.947358 28209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:49.941178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:56643)
0812 17:48:49.941508 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:49.941523 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:49.941620 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:49.941943 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:49.941946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:49.942141 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:49.942243 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.942249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.943114 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.943117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.943783 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:49.943793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.944335 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.944338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:49.944407 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.944960 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:49.944973 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:49.947220 (+ 2247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":120,"threads_started":1}
W20260812 17:48:49.947652 28210 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:49.941348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:56643 (local address 127.13.38.62:42375)
0812 17:48:49.941684 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:49.941689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:49.941710 (+ 21us) server_negotiation.cc:407] Connection header received
0812 17:48:49.941754 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:49.941757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:49.941801 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:49.941898 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:49.942372 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.942983 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.943918 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:49.944173 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:49.944285 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:49.947411 (+ 3126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:56643: BlockingRecv error: recv got EOF from 127.13.38.1:56643 (error 108)
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":166,"threads_started":1}
Aug 12 17:48:50 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:50 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:50.150334 28211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:50.143927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:41829)
0812 17:48:50.144310 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:50.144329 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:50.144477 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:50.144811 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:50.144816 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:50.145123 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:50.145254 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:50.145263 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:50.146240 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:50.146243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:50.146945 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:50.146955 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:50.147397 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:50.147402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:50.147473 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:50.148063 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:50.148076 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:50.150179 (+ 2103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":93,"threads_started":1}
W20260812 17:48:50.150513 28210 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:50.144127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:41829 (local address 127.13.38.62:42375)
0812 17:48:50.144288 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:50.144293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:50.144425 (+ 132us) server_negotiation.cc:407] Connection header received
0812 17:48:50.144593 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:50.144596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:50.144656 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:50.144747 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:50.145412 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:50.146113 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:50.147070 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:50.147282 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:50.147361 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:50.150341 (+ 2980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:41829: BlockingRecv error: recv got EOF from 127.13.38.2:41829 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 17:48:50 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:50 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:50.350322 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:50.344378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:35073)
0812 17:48:50.344860 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:50.344874 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:50.345024 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:50.345324 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:50.345327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:50.345553 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:50.345648 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:50.345654 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:50.346429 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:50.346434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:50.347030 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:50.347039 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:50.347574 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:50.347578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:50.347668 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:50.348182 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:50.348193 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:50.350107 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":99,"threads_started":1}
W20260812 17:48:50.350368 28210 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:50.344584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:35073 (local address 127.13.38.62:42375)
0812 17:48:50.344790 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:50.344795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:50.344956 (+ 161us) server_negotiation.cc:407] Connection header received
0812 17:48:50.345143 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:50.345146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:50.345203 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:50.345296 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:50.345774 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:50.346311 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:50.347158 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:50.347382 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:50.347514 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:50.350256 (+ 2742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:35073: BlockingRecv error: recv got EOF from 127.13.38.3:35073 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 17:48:50 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:50 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:50.955145 28216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:50.948456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:54731)
0812 17:48:50.948776 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:50.948794 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:50.948918 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:50.949189 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:50.949192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:50.949384 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:50.949498 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:50.949504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:50.950383 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:50.950386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:50.951235 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:50.951247 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:50.952028 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:50.952032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:50.952132 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:50.952781 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:50.952802 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:50.954908 (+ 2106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":111,"threads_started":1}
W20260812 17:48:50.955185 28218 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:50.948633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:54731 (local address 127.13.38.62:42375)
0812 17:48:50.948908 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:50.948912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:50.948929 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:48:50.949011 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:50.949013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:50.949059 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:50.949144 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:50.949651 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:50.950195 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:50.951395 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:50.951875 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:50.952014 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:50.955053 (+ 3039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:54731: BlockingRecv error: recv got EOF from 127.13.38.1:54731 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":86,"threads_started":1}
Aug 12 17:48:51 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:51 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:51.157779 28221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:51.151461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:59625)
0812 17:48:51.151882 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:51.151896 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:51.152032 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:51.152356 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:51.152358 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:51.152563 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:51.152667 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:51.152673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:51.153601 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:51.153605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:51.154279 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:51.154300 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:51.154779 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:51.154782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:51.154859 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:51.155432 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:51.155446 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:51.157598 (+ 2152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":192,"threads_started":1}
W20260812 17:48:51.157877 28218 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:51.151576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:59625 (local address 127.13.38.62:42375)
0812 17:48:51.151791 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:51.151796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:51.152014 (+ 218us) server_negotiation.cc:407] Connection header received
0812 17:48:51.152166 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:51.152170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:51.152229 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:51.152344 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:51.152828 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:51.153448 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:51.154424 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:51.154652 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:51.154736 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:51.157772 (+ 3036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:59625: BlockingRecv error: recv got EOF from 127.13.38.2:59625 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 17:48:51 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:51 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:51.357641 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:51.351339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:53173)
0812 17:48:51.351676 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:51.351694 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:51.351845 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:51.352171 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:51.352173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:51.352420 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:51.352529 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:51.352536 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:51.353486 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:51.353489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:51.354082 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:51.354092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:51.354520 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:51.354523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:51.354586 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:51.355152 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:51.355164 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:51.357479 (+ 2315us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":114,"threads_started":1}
W20260812 17:48:51.357797 28218 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:51.351583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:53173 (local address 127.13.38.62:42375)
0812 17:48:51.351759 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:51.351762 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:51.351778 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:48:51.351965 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:51.351967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:51.352019 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:51.352133 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:51.352700 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:51.353327 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:51.354198 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:51.354416 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:51.354500 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:51.357661 (+ 3161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:53173: BlockingRecv error: recv got EOF from 127.13.38.3:53173 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 12 17:48:51 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:51 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:51.962502 28224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:51.956168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:46841)
0812 17:48:51.956522 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:51.956540 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:51.956639 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:51.957026 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:51.957030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:51.957284 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:51.957398 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:51.957405 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:51.958251 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:51.958254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:51.958820 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:51.958829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:51.959384 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:51.959387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:51.959454 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:51.960130 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:51.960142 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:51.962258 (+ 2116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":95,"threads_started":1}
W20260812 17:48:51.962571 28226 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:51.956197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:46841 (local address 127.13.38.62:42375)
0812 17:48:51.956506 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:51.956512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:51.956615 (+ 103us) server_negotiation.cc:407] Connection header received
0812 17:48:51.956760 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:51.956762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:51.956810 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:51.956941 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:51.957567 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:51.958122 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:51.958988 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:51.959256 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:51.959357 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:51.962428 (+ 3071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:46841: BlockingRecv error: recv got EOF from 127.13.38.1:46841 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":93,"threads_started":1}
Aug 12 17:48:52 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:52 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:52.165459 28227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:52.158910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:48859)
0812 17:48:52.159225 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:52.159249 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:52.159364 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:52.159722 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:52.159724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:52.159918 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:52.160026 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:52.160031 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:52.160853 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:52.160856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:52.161428 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:52.161437 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:52.162154 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:52.162157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:52.162287 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:52.162994 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:52.163013 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:52.165308 (+ 2295us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":127,"threads_started":1}
W20260812 17:48:52.165755 28226 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:52.159028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:48859 (local address 127.13.38.62:42375)
0812 17:48:52.159216 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:52.159220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:52.159329 (+ 109us) server_negotiation.cc:407] Connection header received
0812 17:48:52.159497 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:52.159501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:52.159589 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:52.159677 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:52.160173 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:52.160743 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:52.161563 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:52.161967 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:52.162093 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:52.165467 (+ 3374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:48859: BlockingRecv error: recv got EOF from 127.13.38.2:48859 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 12 17:48:52 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:52 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:52.364504 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:52.358657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:53619)
0812 17:48:52.358959 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:52.358972 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:52.359116 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:52.359403 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:52.359406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:52.359652 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:52.359753 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:52.359758 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:52.360528 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:52.360533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:52.361131 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:52.361140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:52.361628 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:52.361631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:52.361719 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:52.362333 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:52.362346 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:52.364298 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":92,"threads_started":1}
W20260812 17:48:52.364569 28226 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:52.358735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:53619 (local address 127.13.38.62:42375)
0812 17:48:52.358907 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:52.358912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:52.359049 (+ 137us) server_negotiation.cc:407] Connection header received
0812 17:48:52.359233 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:52.359243 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:52.359291 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:52.359376 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:52.359879 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:52.360412 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:52.361264 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:52.361490 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:52.361620 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:52.364444 (+ 2824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:53619: BlockingRecv error: recv got EOF from 127.13.38.3:53619 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 12 17:48:52 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:52 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:52.970249 28229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:52.963650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:47691)
0812 17:48:52.964058 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:52.964074 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:52.964181 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:52.964534 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:52.964540 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:52.964786 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:52.964898 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:52.964906 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:52.966176 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:52.966179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:52.966748 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:52.966757 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:52.967315 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:52.967318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:52.967393 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:52.968139 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:52.968151 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:52.970112 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":132,"threads_started":1}
W20260812 17:48:52.970552 28230 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:52.963823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:47691 (local address 127.13.38.62:42375)
0812 17:48:52.964137 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:52.964143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:52.964163 (+ 20us) server_negotiation.cc:407] Connection header received
0812 17:48:52.964271 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:52.964274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:52.964338 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:52.964456 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:52.965224 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:52.966022 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:52.966887 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:52.967162 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:52.967252 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:52.970372 (+ 3120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:47691: BlockingRecv error: recv got EOF from 127.13.38.1:47691 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":106,"threads_started":1}
Aug 12 17:48:53 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:53 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:53.175946 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:53.166473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:57375)
0812 17:48:53.166850 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:53.166863 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:53.167019 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:53.167326 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:53.167329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:53.167628 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:53.167738 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:53.167746 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:53.168552 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:53.168557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:53.169128 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:53.169137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:53.169627 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:53.169631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:53.169719 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:53.173395 (+ 3676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:53.173409 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:53.175737 (+ 2328us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":122,"threads_started":1}
W20260812 17:48:53.175967 28230 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:53.166607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:57375 (local address 127.13.38.62:42375)
0812 17:48:53.166762 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:53.166766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:53.166948 (+ 182us) server_negotiation.cc:407] Connection header received
0812 17:48:53.167133 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:53.167136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:53.167190 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:53.167298 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:53.167871 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:53.168419 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:53.169257 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:53.169501 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:53.169617 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:53.175866 (+ 6249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:57375: BlockingRecv error: recv got EOF from 127.13.38.2:57375 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 17:48:53 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:53 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:53.371682 28233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:53.365683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:60799)
0812 17:48:53.366143 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:53.366161 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:53.366362 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:53.366693 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:53.366696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:53.366886 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:53.367043 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:53.367049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:53.367833 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:53.367837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:53.368421 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:53.368431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:53.368881 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:53.368883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:53.368949 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:53.369470 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:53.369481 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:53.371494 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":98,"threads_started":1}
W20260812 17:48:53.371747 28230 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:53.365784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:60799 (local address 127.13.38.62:42375)
0812 17:48:53.365934 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:53.365941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:53.366299 (+ 358us) server_negotiation.cc:407] Connection header received
0812 17:48:53.366473 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:53.366476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:53.366538 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:53.366655 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:53.367156 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:53.367699 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:53.368542 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:53.368771 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:53.368855 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:53.371645 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:60799: BlockingRecv error: recv got EOF from 127.13.38.3:60799 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 17:48:53 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:53 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:53.977751 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:53.971320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:49049)
0812 17:48:53.971676 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:53.971690 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:53.971786 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:53.972118 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:53.972121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:53.972335 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:53.972445 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:53.972451 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:53.973315 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:53.973319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:53.973944 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:53.973954 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:53.974443 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:53.974446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:53.974520 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:53.975090 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:53.975102 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:53.977629 (+ 2527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":116,"threads_started":1}
W20260812 17:48:53.977960 28238 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:53.971431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:49049 (local address 127.13.38.62:42375)
0812 17:48:53.971738 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:53.971742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:53.971759 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:48:53.971883 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:53.971885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:53.971931 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:53.972024 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:53.972600 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:53.973180 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:53.974071 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:53.974305 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:53.974393 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:53.977773 (+ 3380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:49049: BlockingRecv error: recv got EOF from 127.13.38.1:49049 (error 108)
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":139,"threads_started":1}
Aug 12 17:48:54 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:54 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:54.183557 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:54.177002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:33181)
0812 17:48:54.177453 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:54.177471 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:54.177639 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:54.177942 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:54.177945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:54.178226 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:54.178324 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:54.178331 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:54.179119 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:54.179124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:54.179766 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:54.179775 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:54.180256 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:54.180259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:54.180351 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:54.180948 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:54.180962 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:54.183331 (+ 2369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":181,"threads_started":1}
W20260812 17:48:54.183892 28238 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:54.177233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:33181 (local address 127.13.38.62:42375)
0812 17:48:54.177437 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:54.177443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:54.177569 (+ 126us) server_negotiation.cc:407] Connection header received
0812 17:48:54.177755 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:54.177758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:54.177814 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:54.177911 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:54.178459 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:54.179000 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:54.179898 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:54.180131 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:54.180247 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:54.183613 (+ 3366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:33181: BlockingRecv error: recv got EOF from 127.13.38.2:33181 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 12 17:48:54 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:54 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:54.379015 28242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:54.372719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:42027)
0812 17:48:54.373050 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:54.373069 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:54.373195 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:54.373461 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:54.373463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:54.373678 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:54.373780 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:54.373785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:54.374613 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:54.374616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:54.375208 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:54.375218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:54.375786 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:54.375789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:54.375861 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:54.376479 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:54.376492 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:54.378891 (+ 2399us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":143,"threads_started":1}
W20260812 17:48:54.379251 28238 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:54.372902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:42027 (local address 127.13.38.62:42375)
0812 17:48:54.373052 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:54.373056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:54.373203 (+ 147us) server_negotiation.cc:407] Connection header received
0812 17:48:54.373289 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:54.373292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:54.373344 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:54.373415 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:54.373952 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:54.374492 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:54.375338 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:54.375627 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:54.375731 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:54.379101 (+ 3370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:42027: BlockingRecv error: recv got EOF from 127.13.38.3:42027 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 17:48:54 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:54 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:54.988519 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:54.978655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:58035)
0812 17:48:54.979085 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:54.979101 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:54.979237 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:54.979687 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:54.979690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:54.980011 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:54.980171 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:54.980180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:54.981581 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:54.981587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:54.982834 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:54.982851 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:54.984407 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:54.984413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:54.984538 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:54.985182 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:54.985205 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:54.988337 (+ 3132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":103,"threads_started":1}
W20260812 17:48:54.988818 28245 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:54.978775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:58035 (local address 127.13.38.62:42375)
0812 17:48:54.979085 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:54.979089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:54.979177 (+ 88us) server_negotiation.cc:407] Connection header received
0812 17:48:54.979358 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:54.979361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:54.979450 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:54.979634 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:54.980318 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:54.981441 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:54.983627 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:54.984055 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:54.984167 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:54.988500 (+ 4333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:58035: BlockingRecv error: recv got EOF from 127.13.38.1:58035 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":111,"threads_started":1}
Aug 12 17:48:55 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:55 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:55.191264 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:55.184579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:38551)
0812 17:48:55.184905 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:55.184921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:55.185059 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:55.185388 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:55.185391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:55.185688 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:55.185793 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:55.185800 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:55.186681 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:55.186684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:55.187261 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:55.187270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:55.188037 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:55.188040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:55.188112 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:55.188748 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:55.188761 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:55.191131 (+ 2370us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":90,"threads_started":1}
W20260812 17:48:55.191502 28245 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:55.184620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:38551 (local address 127.13.38.62:42375)
0812 17:48:55.184777 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:55.184790 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:55.185072 (+ 282us) server_negotiation.cc:407] Connection header received
0812 17:48:55.185154 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:55.185156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:55.185221 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:55.185341 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:55.185945 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:55.186555 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:55.187414 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:55.187801 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:55.187932 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:55.191343 (+ 3411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:38551: BlockingRecv error: recv got EOF from 127.13.38.2:38551 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 17:48:55 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:55 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:55.386034 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:55.379828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:44041)
0812 17:48:55.380223 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:55.380238 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:55.380400 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:55.380738 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:55.380741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:55.381041 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:55.381147 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:55.381153 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:55.381955 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:55.381959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:55.382533 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:55.382542 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:55.383024 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:55.383028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:55.383116 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:55.383763 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:55.383775 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:55.385907 (+ 2132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":101,"threads_started":1}
W20260812 17:48:55.386242 28245 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:55.379989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:44041 (local address 127.13.38.62:42375)
0812 17:48:55.380180 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:55.380185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:55.380340 (+ 155us) server_negotiation.cc:407] Connection header received
0812 17:48:55.380531 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:55.380546 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:55.380608 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:55.380708 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:55.381282 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:55.381820 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:55.382665 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:55.382895 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:55.383017 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:55.386072 (+ 3055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:44041: BlockingRecv error: recv got EOF from 127.13.38.3:44041 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 12 17:48:55 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:55 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:55.995328 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:55.989565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:33761)
0812 17:48:55.989920 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:55.989934 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:55.990045 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:55.990332 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:55.990334 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:55.990526 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:55.990631 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:55.990637 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:55.991464 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:55.991467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:55.992128 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:55.992138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:55.992589 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:55.992592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:55.992657 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:55.993181 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:55.993194 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:55.995148 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":100,"threads_started":1}
W20260812 17:48:55.995477 28251 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:55.989632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:33761 (local address 127.13.38.62:42375)
0812 17:48:55.990063 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:55.990067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:55.990085 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:48:55.990128 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:55.990130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:55.990172 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:55.990255 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:55.990788 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:55.991336 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:55.992252 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:55.992478 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:55.992565 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:55.995308 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:33761: BlockingRecv error: recv got EOF from 127.13.38.1:33761 (error 108)
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":99,"threads_started":1}
Aug 12 17:48:56 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:56 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:56.198726 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:56.192343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:43749)
0812 17:48:56.192718 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:56.192751 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:56.192886 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:56.193210 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:56.193216 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:56.193492 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:56.193616 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:56.193631 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:56.194459 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:56.194462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:56.195071 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:56.195081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:56.195629 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:56.195633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:56.195718 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:56.196349 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:56.196372 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:56.198542 (+ 2170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":113,"threads_started":1}
W20260812 17:48:56.198812 28251 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:56.192467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:43749 (local address 127.13.38.62:42375)
0812 17:48:56.192659 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:56.192665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:56.192832 (+ 167us) server_negotiation.cc:407] Connection header received
0812 17:48:56.193010 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:56.193013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:56.193073 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:56.193163 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:56.193792 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:56.194341 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:56.195197 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:56.195419 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:56.195502 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:56.198692 (+ 3190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:43749: BlockingRecv error: recv got EOF from 127.13.38.2:43749 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 17:48:56 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:56 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:56.393231 28257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:56.386824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:56097)
0812 17:48:56.387147 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:56.387161 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:56.387262 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:56.387509 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:56.387511 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:56.387773 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:56.387921 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:56.387929 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:56.389024 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:56.389027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:56.389772 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:56.389787 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:56.390292 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:56.390296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:56.390375 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:56.390947 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:56.390960 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:56.393107 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":99,"threads_started":1}
W20260812 17:48:56.393386 28251 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:56.386954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:56097 (local address 127.13.38.62:42375)
0812 17:48:56.387109 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:56.387113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:56.387235 (+ 122us) server_negotiation.cc:407] Connection header received
0812 17:48:56.387344 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:56.387347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:56.387394 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:56.387485 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:56.388043 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:56.388891 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:56.389923 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:56.390161 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:56.390252 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:56.393257 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:56097: BlockingRecv error: recv got EOF from 127.13.38.3:56097 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 17:48:56 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:56 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:57.002683 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:56.996403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:45167)
0812 17:48:56.996917 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:56.996931 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:56.997037 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:56.997367 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:56.997370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:56.997609 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:56.997713 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:56.997744 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:56.998571 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:56.998575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:56.999203 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:56.999218 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:56.999684 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:56.999688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:56.999780 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:57.000543 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:57.000556 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:57.002553 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":122,"threads_started":1}
W20260812 17:48:57.002983 28259 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:56.996604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:45167 (local address 127.13.38.62:42375)
0812 17:48:56.996882 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:56.996887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:56.997019 (+ 132us) server_negotiation.cc:407] Connection header received
0812 17:48:56.997141 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:56.997144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:56.997211 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:56.997332 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:56.997885 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:56.998445 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:56.999323 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:56.999568 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:56.999646 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:57.002829 (+ 3183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:45167: BlockingRecv error: recv got EOF from 127.13.38.1:45167 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
Aug 12 17:48:57 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:57 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:57.206266 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:57.199832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:34351)
0812 17:48:57.200220 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:57.200238 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:57.200368 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:57.200712 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:57.200715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:57.201011 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:57.201120 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:57.201127 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:57.202227 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:57.202231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:57.202858 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:57.202867 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:57.203392 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:57.203395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:57.203467 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:57.204086 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:57.204099 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:57.206129 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":130,"threads_started":1}
W20260812 17:48:57.206413 28259 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:57.200062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:34351 (local address 127.13.38.62:42375)
0812 17:48:57.200217 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:57.200222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:57.200333 (+ 111us) server_negotiation.cc:407] Connection header received
0812 17:48:57.200473 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:57.200476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:57.200542 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:57.200680 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:57.201274 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:57.202094 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:57.203021 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:57.203245 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:57.203331 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:57.206289 (+ 2958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:34351: BlockingRecv error: recv got EOF from 127.13.38.2:34351 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 17:48:57 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:57 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:57.399633 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:57.394110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:56699)
0812 17:48:57.394402 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:57.394419 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:57.394548 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:57.394782 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:57.394784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:57.394978 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:57.395103 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:57.395109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:57.395908 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:57.395911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:57.396485 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:57.396493 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:57.396962 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:57.396965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:57.397028 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:57.397533 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:57.397544 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:57.399440 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":108,"threads_started":1}
W20260812 17:48:57.399749 28259 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:57.394198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:56699 (local address 127.13.38.62:42375)
0812 17:48:57.394362 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:57.394366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:57.394482 (+ 116us) server_negotiation.cc:407] Connection header received
0812 17:48:57.394638 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:57.394640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:57.394687 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:57.394755 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:57.395208 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:57.395806 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:57.396630 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:57.396847 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:57.396940 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:57.399623 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:56699: BlockingRecv error: recv got EOF from 127.13.38.3:56699 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 17:48:58 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:58 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:58.010182 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:58.003827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:34619)
0812 17:48:58.004166 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:58.004180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:58.004286 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:58.004582 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:58.004585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:58.004815 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:58.004918 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:58.004924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:58.005774 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:58.005776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:58.006334 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:58.006342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:58.006821 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:58.006824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:58.006889 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:58.007456 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:58.007468 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:58.010036 (+ 2568us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":111,"threads_started":1}
W20260812 17:48:58.010375 28266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:58.003972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:34619 (local address 127.13.38.62:42375)
0812 17:48:58.004276 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:58.004280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:58.004299 (+ 19us) server_negotiation.cc:407] Connection header received
0812 17:48:58.004354 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:58.004356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:58.004400 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:58.004500 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:58.005078 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:58.005659 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:58.006453 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:58.006706 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:58.006790 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:58.010192 (+ 3402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:34619: BlockingRecv error: recv got EOF from 127.13.38.1:34619 (error 108)
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":128,"threads_started":1}
Aug 12 17:48:58 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:58 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:58.216508 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:58.207224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:33815)
0812 17:48:58.207591 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:58.207605 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:58.207707 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:58.207949 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:58.207951 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:58.208142 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:58.208239 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:58.208244 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:58.208980 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:58.208982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:58.209555 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:58.209565 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:58.209985 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:58.209987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:58.210050 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:58.213780 (+ 3730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:58.213795 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:58.216324 (+ 2529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":142,"threads_started":1}
W20260812 17:48:58.216761 28266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:58.207283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:33815 (local address 127.13.38.62:42375)
0812 17:48:58.207441 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:58.207445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:58.207678 (+ 233us) server_negotiation.cc:407] Connection header received
0812 17:48:58.207791 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:58.207793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:58.207839 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:58.207913 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:58.208354 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:58.208871 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:58.209670 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:58.209884 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:58.209964 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:58.216610 (+ 6646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:33815: BlockingRecv error: recv got EOF from 127.13.38.2:33815 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 17:48:58 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:58 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:58.406653 28268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:58.400655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:39097)
0812 17:48:58.401005 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:58.401018 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:58.401164 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:58.401500 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:58.401503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:58.401734 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:58.401828 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:58.401834 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:58.402659 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:58.402664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:58.403229 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:58.403238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:58.403782 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:58.403785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:58.403871 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:58.404473 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:58.404486 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:58.406522 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":145,"threads_started":1}
W20260812 17:48:58.406900 28266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:58.400722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:39097 (local address 127.13.38.62:42375)
0812 17:48:58.400895 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:58.400901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:58.401104 (+ 203us) server_negotiation.cc:407] Connection header received
0812 17:48:58.401297 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:58.401302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:58.401369 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:58.401471 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:58.401954 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:58.402539 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:58.403360 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:58.403660 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:58.403774 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:58.406757 (+ 2983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:39097: BlockingRecv error: recv got EOF from 127.13.38.3:39097 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 17:48:59 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:59 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:59.017483 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:59.011126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:45293)
0812 17:48:59.011438 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:59.011463 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:59.011591 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:59.011856 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:59.011858 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:59.012098 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:59.012207 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:59.012213 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:59.013088 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:59.013092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:59.013674 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:59.013684 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:59.014245 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:59.014247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:59.014319 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:59.014872 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:59.014884 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:59.017327 (+ 2443us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":126,"threads_started":1}
W20260812 17:48:59.017866 28270 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:59.011258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:45293 (local address 127.13.38.62:42375)
0812 17:48:59.011508 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:59.011512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:59.011585 (+ 73us) server_negotiation.cc:407] Connection header received
0812 17:48:59.011680 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:59.011682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:59.011726 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:59.011813 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:59.012348 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:59.012941 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:59.013814 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:59.014107 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:59.014208 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:59.017555 (+ 3347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:45293: BlockingRecv error: recv got EOF from 127.13.38.1:45293 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":91,"threads_started":1}
Aug 12 17:48:59 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:59 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:59.223337 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:59.217487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:38389)
0812 17:48:59.217763 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:59.217779 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:59.217887 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:59.218209 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:59.218212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:59.218408 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:59.218509 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:59.218515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:59.219385 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:59.219388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:59.220159 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:59.220171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:59.220653 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:59.220656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:59.220726 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:59.221253 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:59.221265 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:59.223186 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":94,"threads_started":1}
W20260812 17:48:59.223604 28270 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:59.217554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:38389 (local address 127.13.38.62:42375)
0812 17:48:59.217721 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:59.217725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:59.217879 (+ 154us) server_negotiation.cc:407] Connection header received
0812 17:48:59.217986 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:59.217988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:59.218039 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:59.218154 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:59.218648 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:59.219230 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:59.220293 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:59.220515 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:59.220617 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:59.223406 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:38389: BlockingRecv error: recv got EOF from 127.13.38.2:38389 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 17:48:59 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:48:59 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:48:59.413932 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:48:59.407764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:53913)
0812 17:48:59.408157 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0812 17:48:59.408180 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 17:48:59.408285 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:48:59.408559 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:48:59.408562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:48:59.408757 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:48:59.408859 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:59.408864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:59.409849 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:59.409852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:59.410575 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:48:59.410587 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:48:59.411110 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:48:59.411113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:48:59.411210 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:59.411795 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:48:59.411807 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:48:59.413723 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":203,"threads_started":1}
W20260812 17:48:59.413980 28270 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:48:59.407948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:53913 (local address 127.13.38.62:42375)
0812 17:48:59.408148 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 17:48:59.408153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:48:59.408256 (+ 103us) server_negotiation.cc:407] Connection header received
0812 17:48:59.408376 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:48:59.408378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:48:59.408426 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:48:59.408508 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:48:59.408985 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:59.409699 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:59.410716 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:48:59.410953 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:48:59.411101 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:48:59.413869 (+ 2768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:53913: BlockingRecv error: recv got EOF from 127.13.38.3:53913 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 12 17:49:00 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:00 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:00.024405 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:00.018345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:57077)
0812 17:49:00.018692 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:00.018709 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:00.018809 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:00.019129 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:00.019131 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:00.019337 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:00.019466 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:00.019472 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:00.020363 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:00.020366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:00.020970 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:00.020983 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:00.021464 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:00.021467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:00.021538 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:00.022032 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:00.022044 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:00.024222 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":122,"threads_started":1}
W20260812 17:49:00.024513 28275 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:00.018522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:57077 (local address 127.13.38.62:42375)
0812 17:49:00.018751 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:00.018755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:00.018769 (+ 14us) server_negotiation.cc:407] Connection header received
0812 17:49:00.018935 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:00.018938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:00.018989 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:00.019070 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:00.019642 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:00.020224 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:00.021112 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:00.021339 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:00.021417 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:00.024372 (+ 2955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:57077: BlockingRecv error: recv got EOF from 127.13.38.1:57077 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":75,"threads_started":1}
Aug 12 17:49:00 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:00 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:00.230260 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:00.224215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:52879)
0812 17:49:00.224521 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:00.224536 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:00.224641 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:00.224889 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:00.224891 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:00.225105 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:00.225205 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:00.225211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:00.226010 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:00.226013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:00.226598 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:00.226607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:00.227118 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:00.227122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:00.227195 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:00.227778 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:00.227791 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:00.230071 (+ 2280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":75,"threads_started":1}
W20260812 17:49:00.230367 28275 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:00.224351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:52879 (local address 127.13.38.62:42375)
0812 17:49:00.224525 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:00.224529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:00.224613 (+ 84us) server_negotiation.cc:407] Connection header received
0812 17:49:00.224725 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:00.224727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:00.224774 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:00.224842 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:00.225342 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:00.225876 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:00.226727 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:00.226949 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:00.227037 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:00.230245 (+ 3208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:52879: BlockingRecv error: recv got EOF from 127.13.38.2:52879 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 12 17:49:00 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:00 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:00.421772 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:00.415056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:46267)
0812 17:49:00.415420 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:00.415436 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:00.415594 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:00.415884 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:00.415886 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:00.416095 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:00.416205 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:00.416211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:00.417165 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:00.417169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:00.418134 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:00.418146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:00.418756 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:00.418760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:00.418857 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:00.419404 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:00.419417 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:00.421638 (+ 2221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":144,"threads_started":1}
W20260812 17:49:00.422206 28275 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:00.415179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:46267 (local address 127.13.38.62:42375)
0812 17:49:00.415384 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:00.415389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:00.415555 (+ 166us) server_negotiation.cc:407] Connection header received
0812 17:49:00.415694 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:00.415696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:00.415750 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:00.415834 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:00.416367 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:00.416998 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:00.418275 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:00.418612 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:00.418737 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:00.421877 (+ 3140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:46267: BlockingRecv error: recv got EOF from 127.13.38.3:46267 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 12 17:49:01 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:01 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 17:49:01.032438 28279 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:01.025569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:35565 (local address 127.13.38.62:42375)
0812 17:49:01.025825 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:01.025829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:01.025845 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:49:01.025892 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:01.025894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:01.025949 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:01.026046 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:01.026558 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:01.027250 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:01.028400 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:01.028799 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:01.028950 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:01.032321 (+ 3371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:35565: BlockingRecv error: recv got EOF from 127.13.38.1:35565 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":95,"threads_started":1}
I20260812 17:49:01.032528 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:01.025414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:35565)
0812 17:49:01.025710 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:01.025725 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:01.025822 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:01.026080 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:01.026105 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:01.026290 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:01.026396 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:01.026402 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:01.027376 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:01.027380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:01.028232 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:01.028246 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:01.028961 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:01.028967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:01.029097 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:01.029791 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:01.029805 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:01.032149 (+ 2344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
Aug 12 17:49:01 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:01 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:01.238027 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:01.231581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:47971)
0812 17:49:01.231960 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:01.231974 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:01.232124 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:01.232440 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:01.232443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:01.232681 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:01.232801 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:01.232807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:01.233603 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:01.233607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:01.234181 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:01.234189 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:01.234683 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:01.234687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:01.234799 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:01.235351 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:01.235364 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:01.237829 (+ 2465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":112,"threads_started":1}
W20260812 17:49:01.238344 28279 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:01.231693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:47971 (local address 127.13.38.62:42375)
0812 17:49:01.231902 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:01.231908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:01.232058 (+ 150us) server_negotiation.cc:407] Connection header received
0812 17:49:01.232250 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:01.232253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:01.232314 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:01.232409 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:01.232922 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:01.233474 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:01.234306 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:01.234552 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:01.234665 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:01.238056 (+ 3391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:47971: BlockingRecv error: recv got EOF from 127.13.38.2:47971 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 12 17:49:01 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:01 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:01.429229 28283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:01.422709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:43913)
0812 17:49:01.423001 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:01.423018 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:01.423134 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:01.423470 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:01.423476 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:01.423773 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:01.423887 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:01.423896 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:01.424785 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:01.424788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:01.425361 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:01.425371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:01.425898 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:01.425901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:01.425980 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:01.426712 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:01.426727 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:01.429105 (+ 2378us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":114,"threads_started":1}
W20260812 17:49:01.429386 28279 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:01.422829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:43913 (local address 127.13.38.62:42375)
0812 17:49:01.422976 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:01.422979 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:01.423083 (+ 104us) server_negotiation.cc:407] Connection header received
0812 17:49:01.423206 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:01.423209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:01.423256 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:01.423332 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:01.424075 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:01.424650 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:01.425506 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:01.425752 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:01.425840 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:01.429253 (+ 3413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:43913: BlockingRecv error: recv got EOF from 127.13.38.3:43913 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 17:49:02 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:02 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:02.039995 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:02.033421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:43091)
0812 17:49:02.033758 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:02.033774 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:02.033941 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:02.034282 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:02.034284 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:02.034492 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:02.034599 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:02.034604 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:02.035430 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:02.035434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:02.036089 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:02.036099 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:02.036622 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:02.036625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:02.036720 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:02.037408 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:02.037421 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:02.039797 (+ 2376us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":106,"threads_started":1}
W20260812 17:49:02.040124 28285 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:02.033429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:43091 (local address 127.13.38.62:42375)
0812 17:49:02.033749 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:02.033754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:02.033884 (+ 130us) server_negotiation.cc:407] Connection header received
0812 17:49:02.034106 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:02.034108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:02.034157 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:02.034237 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:02.034747 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:02.035298 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:02.036221 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:02.036478 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:02.036582 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:02.039971 (+ 3389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:43091: BlockingRecv error: recv got EOF from 127.13.38.1:43091 (error 108)
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":138,"threads_started":1}
Aug 12 17:49:02 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:02 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:02.246744 28286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:02.239843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:45721)
0812 17:49:02.240397 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:02.240416 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:02.240572 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:02.240974 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:02.240977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:02.241227 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:02.241372 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:02.241381 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:02.242685 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:02.242689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:02.243310 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:02.243319 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:02.243902 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:02.243906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:02.243994 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:02.244542 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:02.244555 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:02.246532 (+ 1977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":120,"threads_started":1}
W20260812 17:49:02.246827 28285 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:02.239837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:45721 (local address 127.13.38.62:42375)
0812 17:49:02.240021 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:02.240026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:02.240500 (+ 474us) server_negotiation.cc:407] Connection header received
0812 17:49:02.240694 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:02.240698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:02.240758 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:02.240850 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:02.241502 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:02.242445 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:02.243444 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:02.243769 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:02.243893 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:02.246692 (+ 2799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:45721: BlockingRecv error: recv got EOF from 127.13.38.2:45721 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 17:49:02 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:02 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:02.436482 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:02.430336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:42953)
0812 17:49:02.430689 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:02.430703 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:02.430815 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:02.431197 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:02.431199 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:02.431409 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:02.431514 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:02.431545 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:02.432375 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:02.432395 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:02.432964 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:02.432974 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:02.433448 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:02.433450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:02.433516 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:02.434319 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:02.434331 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:02.436361 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":119,"threads_started":1}
W20260812 17:49:02.436705 28285 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:02.430406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:42953 (local address 127.13.38.62:42375)
0812 17:49:02.430605 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:02.430611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:02.430801 (+ 190us) server_negotiation.cc:407] Connection header received
0812 17:49:02.430927 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:02.430930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:02.431048 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:02.431171 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:02.431683 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:02.432233 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:02.433086 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:02.433305 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:02.433388 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:02.436538 (+ 3150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:42953: BlockingRecv error: recv got EOF from 127.13.38.3:42953 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 12 17:49:03 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:03 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:03.047183 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:03.041236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:47351)
0812 17:49:03.041633 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:03.041654 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:03.041784 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:03.042111 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:03.042114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:03.042335 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:03.042439 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:03.042446 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:03.043239 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:03.043244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:03.043854 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:03.043863 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:03.044359 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:03.044362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:03.044456 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:03.045004 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:03.045016 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:03.047051 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":95,"threads_started":1}
W20260812 17:49:03.047402 28290 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:03.041486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:47351 (local address 127.13.38.62:42375)
0812 17:49:03.041837 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:03.041841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:03.041855 (+ 14us) server_negotiation.cc:407] Connection header received
0812 17:49:03.041897 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:03.041899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:03.041947 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:03.042074 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:03.042576 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:03.043117 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:03.043993 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:03.044229 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:03.044342 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:03.047205 (+ 2863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:47351: BlockingRecv error: recv got EOF from 127.13.38.1:47351 (error 108)
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":84,"threads_started":1}
Aug 12 17:49:03 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:03 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:03.260020 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:03.247819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:60965)
0812 17:49:03.248205 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:03.248220 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:03.248352 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:03.248675 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:03.248678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:03.248926 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:03.249067 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:03.249076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:03.249837 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:03.249839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:03.250426 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:03.250435 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:03.250927 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:03.250931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:03.251008 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:03.257614 (+ 6606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:03.257630 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:03.259862 (+ 2232us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":94,"threads_started":1}
W20260812 17:49:03.260242 28290 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:03.248018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:60965 (local address 127.13.38.62:42375)
0812 17:49:03.248162 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:03.248168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:03.248307 (+ 139us) server_negotiation.cc:407] Connection header received
0812 17:49:03.248478 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:03.248481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:03.248544 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:03.248640 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:03.249191 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:03.249739 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:03.250545 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:03.250768 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:03.250950 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:03.260086 (+ 9136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:60965: BlockingRecv error: recv got EOF from 127.13.38.2:60965 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 17:49:03 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:03 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.3@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:03.443877 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:03.437326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.3:59571)
0812 17:49:03.437613 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:03.437626 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:03.437728 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:03.438020 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:03.438023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:03.438273 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:03.438403 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:03.438410 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:03.439567 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:03.439572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:03.440236 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:03.440247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:03.440755 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:03.440758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:03.440829 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:03.441416 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:03.441429 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:03.443711 (+ 2282us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":100,"threads_started":1}
W20260812 17:49:03.444121 28290 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:03.437400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:59571 (local address 127.13.38.62:42375)
0812 17:49:03.437586 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:03.437591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:03.437720 (+ 129us) server_negotiation.cc:407] Connection header received
0812 17:49:03.437795 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:03.437797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:03.437853 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:03.437928 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:03.438627 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:03.439361 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:03.440392 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:03.440632 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:03.440729 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:03.443931 (+ 3202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.3:59571: BlockingRecv error: recv got EOF from 127.13.38.3:59571 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 12 17:49:04 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:04 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.1@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:04.054358 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:04.048102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.1:52297)
0812 17:49:04.048425 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:04.048440 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:04.048534 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:04.048833 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:04.048835 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:04.049061 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:04.049161 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:04.049167 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.050017 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.050020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:04.050612 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:04.050621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.051172 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.051175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:04.051290 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:04.051960 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:04.051972 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:04.054221 (+ 2249us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":89,"threads_started":1}
W20260812 17:49:04.054720 28297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:04.048236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:52297 (local address 127.13.38.62:42375)
0812 17:49:04.048492 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:04.048496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:04.048509 (+ 13us) server_negotiation.cc:407] Connection header received
0812 17:49:04.048661 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:04.048664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:04.048712 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:04.048800 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:04.049310 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.049870 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.050738 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.050996 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.051105 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:04.054391 (+ 3286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.1:52297: BlockingRecv error: recv got EOF from 127.13.38.1:52297 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":63,"threads_started":1}
Aug 12 17:49:04 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 17:49:04 dist-test-slave-sb2z krb5kdc[26603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.38.2@KRBTEST.COM for oryx/127.13.38.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 17:49:04.267083 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:04.261146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42375 (local address 127.13.38.2:60447)
0812 17:49:04.261523 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:04.261537 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:04.261641 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:04.261891 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:04.261893 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:04.262106 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:04.262206 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:04.262212 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.262955 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.262958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:04.263577 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:04.263590 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.264024 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.264027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:04.264107 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:04.264706 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:04.264718 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:04.266927 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.38.62:42375: Server oryx/127.13.38.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":117,"threads_started":1}
W20260812 17:49:04.267167 28297 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:04.261291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:60447 (local address 127.13.38.62:42375)
0812 17:49:04.261463 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:04.261467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:04.261611 (+ 144us) server_negotiation.cc:407] Connection header received
0812 17:49:04.261726 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:04.261728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:04.261773 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:04.261857 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:04.262316 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.262857 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.263692 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.263910 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.264003 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:04.267070 (+ 3067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.38.2:60447: BlockingRecv error: recv got EOF from 127.13.38.2:60447 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 17:49:04.325148 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 27539
I20260812 17:49:04.340011 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 27780
I20260812 17:49:04.351382 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 27992
I20260812 17:49:04.364171 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 27421
2026-08-12T17:49:04Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18400 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T17:49:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:49:04Z Disabled control of system clock
I20260812 17:49:04.407826 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34529
--webserver_interface=127.13.38.62
--webserver_port=0
--builtin_ntp_servers=127.13.38.20:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 17:49:04.529477 28306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:04.529785 28306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:04.529822 28306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:04.534142 28306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:49:04.534215 28306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:04.534230 28306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:04.534247 28306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:49:04.534263 28306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:49:04.539393 28306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:34529
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:04.540706 28306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:04.542237 28306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 17:49:04.551983 28306 server_base.cc:1034] running on GCE node
W20260812 17:49:04.551977 28317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:04.551980 28318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:04.552013 28320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:04.552855 28306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:04.553972 28306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:04.555153 28306 hybrid_clock.cc:648] HybridClock initialized: now 1786556944555124 us; error 52 us; skew 500 ppm
I20260812 17:49:04.557204 28306 webserver.cc:460] Webserver started at http://127.13.38.62:42547/ using document root <none> and password file <none>
I20260812 17:49:04.557829 28306 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:04.557895 28306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:04.558143 28306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:49:04.560437 28306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "25bf6e1124674a828ee16bccf08153dc"
format_stamp: "Formatted at 2026-08-12 17:49:04 on dist-test-slave-sb2z"
server_key: "e80b13c9be23bf18219315b4ecaccea7"
server_key_iv: "837f76eff40c3419a4bf03274fe4bb32"
server_key_version: "encryptionkey@0"
I20260812 17:49:04.561092 28306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "25bf6e1124674a828ee16bccf08153dc"
format_stamp: "Formatted at 2026-08-12 17:49:04 on dist-test-slave-sb2z"
server_key: "e80b13c9be23bf18219315b4ecaccea7"
server_key_iv: "837f76eff40c3419a4bf03274fe4bb32"
server_key_version: "encryptionkey@0"
I20260812 17:49:04.565138 28306 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:49:04.567942 28333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:04.569250 28306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:49:04.569401 28306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "25bf6e1124674a828ee16bccf08153dc"
format_stamp: "Formatted at 2026-08-12 17:49:04 on dist-test-slave-sb2z"
server_key: "e80b13c9be23bf18219315b4ecaccea7"
server_key_iv: "837f76eff40c3419a4bf03274fe4bb32"
server_key_version: "encryptionkey@0"
I20260812 17:49:04.569525 28306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:04.579764 28306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:04.580466 28306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:04.580672 28306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:04.588249 28306 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:34529
I20260812 17:49:04.588263 28423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:34529 every 8 connection(s)
I20260812 17:49:04.589278 28306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:49:04.592952 28424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:49:04.595209 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 28306
I20260812 17:49:04.595331 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:49:04.595773 13464 external_mini_cluster.cc:1428] Setting key c22139e3940995320bb93f9ec686e48d
I20260812 17:49:04.600381 28424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc: Bootstrap starting.
I20260812 17:49:04.603415 28424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc: Neither blocks nor log segments found. Creating new log.
I20260812 17:49:04.604453 28424 log.cc:824] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc: Log is configured to *not* fsync() on all Append() calls
I20260812 17:49:04.607208 28424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc: No bootstrap required, opened a new log
I20260812 17:49:04.607873 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:04.597514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52198 (local address 127.13.38.62:34529)
0812 17:49:04.598304 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:04.598343 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:04.598392 (+ 49us) server_negotiation.cc:407] Connection header received
0812 17:49:04.599311 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:04.599337 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:04.599915 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:04.600317 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:04.600826 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.602491 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.603448 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.604245 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.604436 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:04.605397 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:04.605403 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:04.605418 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:49:04.606054 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:04.606133 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:49:04.606162 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:04.606272 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:04.606433 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 17:49:04.606740 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":138,"threads_started":1}
I20260812 17:49:04.611816 28424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bf6e1124674a828ee16bccf08153dc" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34529 } }
I20260812 17:49:04.612079 28424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:49:04.612120 28424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25bf6e1124674a828ee16bccf08153dc, State: Initialized, Role: FOLLOWER
I20260812 17:49:04.612572 28424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bf6e1124674a828ee16bccf08153dc" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34529 } }
I20260812 17:49:04.612712 28424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:49:04.612764 28424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:49:04.612844 28424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:49:04.613898 28424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bf6e1124674a828ee16bccf08153dc" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34529 } }
I20260812 17:49:04.614234 28424 leader_election.cc:302] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25bf6e1124674a828ee16bccf08153dc; no voters:
I20260812 17:49:04.614524 28424 leader_election.cc:288] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:49:04.614732 28431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:49:04.615101 28431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [term 1 LEADER]: Becoming Leader. State: Replica: 25bf6e1124674a828ee16bccf08153dc, State: Running, Role: LEADER
I20260812 17:49:04.615485 28431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bf6e1124674a828ee16bccf08153dc" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34529 } }
I20260812 17:49:04.616078 28424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:49:04.617223 28433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25bf6e1124674a828ee16bccf08153dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bf6e1124674a828ee16bccf08153dc" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34529 } } }
I20260812 17:49:04.617377 28433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [sys.catalog]: This master's current role is: LEADER
I20260812 17:49:04.617811 28445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:49:04.617854 28434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25bf6e1124674a828ee16bccf08153dc. Latest consensus state: current_term: 1 leader_uuid: "25bf6e1124674a828ee16bccf08153dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bf6e1124674a828ee16bccf08153dc" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34529 } } }
I20260812 17:49:04.618227 28434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc [sys.catalog]: This master's current role is: LEADER
I20260812 17:49:04.620903 28445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:49:04.627416 28445 catalog_manager.cc:1347] Generated new cluster ID: dc66544be70c4bc9926c5cacf2f3c3ee
I20260812 17:49:04.627511 28445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:49:04.659044 28445 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:49:04.660414 28445 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:49:04.669420 28445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25bf6e1124674a828ee16bccf08153dc: Generated new TSK 0
I20260812 17:49:04.670333 28445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:49:04.681141 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34529
--builtin_ntp_servers=127.13.38.20:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 17:49:04.805212 28460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:04.805512 28460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:04.805550 28460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:04.809855 28460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:04.809938 28460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:04.809996 28460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:49:04.814990 28460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:04.816251 28460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:04.817937 28460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:04.828568 28467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:04.828570 28472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:04.828812 28460 server_base.cc:1034] running on GCE node
W20260812 17:49:04.828568 28468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:04.829392 28460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:04.830153 28460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:04.831357 28460 hybrid_clock.cc:648] HybridClock initialized: now 1786556944831328 us; error 40 us; skew 500 ppm
I20260812 17:49:04.837749 28460 webserver.cc:460] Webserver started at http://127.13.38.1:36737/ using document root <none> and password file <none>
I20260812 17:49:04.838487 28460 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:04.838565 28460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:04.838878 28460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:49:04.841315 28460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "f5557649baf3498aab3409ac2758d8c3"
format_stamp: "Formatted at 2026-08-12 17:49:04 on dist-test-slave-sb2z"
server_key: "19bc441bf2ba21bc76e7a0a90c7913af"
server_key_iv: "6baf608197d98af2b142e849b6a997cd"
server_key_version: "encryptionkey@0"
I20260812 17:49:04.841950 28460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "f5557649baf3498aab3409ac2758d8c3"
format_stamp: "Formatted at 2026-08-12 17:49:04 on dist-test-slave-sb2z"
server_key: "19bc441bf2ba21bc76e7a0a90c7913af"
server_key_iv: "6baf608197d98af2b142e849b6a997cd"
server_key_version: "encryptionkey@0"
I20260812 17:49:04.846102 28460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:49:04.848912 28480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:04.850255 28460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 17:49:04.850383 28460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "f5557649baf3498aab3409ac2758d8c3"
format_stamp: "Formatted at 2026-08-12 17:49:04 on dist-test-slave-sb2z"
server_key: "19bc441bf2ba21bc76e7a0a90c7913af"
server_key_iv: "6baf608197d98af2b142e849b6a997cd"
server_key_version: "encryptionkey@0"
I20260812 17:49:04.850497 28460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:04.876857 28460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:04.877677 28460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:04.877910 28460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:04.878574 28460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:49:04.879678 28460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:49:04.879727 28460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:04.879796 28460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:49:04.879827 28460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:04.889087 28460 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:46075
I20260812 17:49:04.889107 28660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:46075 every 8 connection(s)
I20260812 17:49:04.890148 28460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:49:04.899878 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:04.892172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:55341 (local address 127.13.38.62:34529)
0812 17:49:04.892351 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:04.892356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:04.893469 (+ 1113us) server_negotiation.cc:407] Connection header received
0812 17:49:04.894852 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:04.894854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:04.894902 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:04.894981 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:04.896072 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.896618 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.897849 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.898080 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.898163 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:04.899295 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:04.899298 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:04.899300 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:49:04.899315 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:04.899334 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:49:04.899342 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:04.899400 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:04.899623 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 17:49:04.899701 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 17:49:04.899865 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 28460
I20260812 17:49:04.899991 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:49:04.900302 13464 external_mini_cluster.cc:1428] Setting key 33966e31d8900b965ccd8a8326533985
I20260812 17:49:04.900936 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:04.892615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34529 (local address 127.13.38.1:55341)
0812 17:49:04.893312 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:04.893355 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:04.894652 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:04.895174 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:04.895181 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:04.895268 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:04.895882 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:04.895905 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.896740 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.896742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:04.897726 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:04.897737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:04.898224 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:04.898227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:04.898372 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:04.899000 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:49:04.899002 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:04.899044 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:49:04.899137 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:49:04.899418 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:49:04.899425 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:49:04.899429 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 17:49:04.899550 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 17:49:04.899868 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":112,"threads_started":1}
I20260812 17:49:04.902395 28661 heartbeater.cc:347] Connected to a master server at 127.13.38.62:34529
I20260812 17:49:04.902627 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34529
--builtin_ntp_servers=127.13.38.20:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 17:49:04.902660 28661 heartbeater.cc:464] Registering TS with master...
I20260812 17:49:04.903236 28661 heartbeater.cc:511] Master 127.13.38.62:34529 requested a full tablet report, sending...
I20260812 17:49:04.904989 28354 ts_manager.cc:194] Registered new tserver with Master: f5557649baf3498aab3409ac2758d8c3 (127.13.38.1:46075)
I20260812 17:49:04.906180 28354 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.38.1:55341
W20260812 17:49:05.030660 28668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:05.031018 28668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:05.031095 28668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:05.038321 28668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:05.038420 28668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:05.038535 28668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:49:05.048807 28668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:05.050609 28668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:05.052651 28668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:05.066509 28679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:05.066565 28674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:05.066872 28668 server_base.cc:1034] running on GCE node
W20260812 17:49:05.066901 28676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:05.067437 28668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:05.068209 28668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:05.069388 28668 hybrid_clock.cc:648] HybridClock initialized: now 1786556945069359 us; error 41 us; skew 500 ppm
I20260812 17:49:05.071776 28668 webserver.cc:460] Webserver started at http://127.13.38.2:38133/ using document root <none> and password file <none>
I20260812 17:49:05.072439 28668 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:05.072528 28668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:05.072813 28668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:49:05.075335 28668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "9c3fe4604c3c4d84b98c60f629b6be4d"
format_stamp: "Formatted at 2026-08-12 17:49:05 on dist-test-slave-sb2z"
server_key: "707952700138a89e7a6857da994bfb15"
server_key_iv: "76aa20e2a7de84dbde8e4c3a740a9249"
server_key_version: "encryptionkey@0"
I20260812 17:49:05.076076 28668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "9c3fe4604c3c4d84b98c60f629b6be4d"
format_stamp: "Formatted at 2026-08-12 17:49:05 on dist-test-slave-sb2z"
server_key: "707952700138a89e7a6857da994bfb15"
server_key_iv: "76aa20e2a7de84dbde8e4c3a740a9249"
server_key_version: "encryptionkey@0"
I20260812 17:49:05.081423 28668 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 17:49:05.084614 28685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:05.086082 28668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:49:05.086233 28668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "9c3fe4604c3c4d84b98c60f629b6be4d"
format_stamp: "Formatted at 2026-08-12 17:49:05 on dist-test-slave-sb2z"
server_key: "707952700138a89e7a6857da994bfb15"
server_key_iv: "76aa20e2a7de84dbde8e4c3a740a9249"
server_key_version: "encryptionkey@0"
I20260812 17:49:05.086371 28668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:05.100086 28668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:05.100835 28668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:05.101070 28668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:05.101718 28668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:49:05.102820 28668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:49:05.102870 28668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:05.102939 28668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:49:05.102972 28668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:05.112321 28668 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:38023
I20260812 17:49:05.112344 28866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:38023 every 8 connection(s)
I20260812 17:49:05.113317 28668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:49:05.122653 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 28668
I20260812 17:49:05.122800 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:49:05.123189 13464 external_mini_cluster.cc:1428] Setting key 5a53785a2b1282b450427df0b361d13f
I20260812 17:49:05.123202 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:05.115328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:42701 (local address 127.13.38.62:34529)
0812 17:49:05.115472 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:05.115476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:05.116747 (+ 1271us) server_negotiation.cc:407] Connection header received
0812 17:49:05.118239 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:05.118242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:05.118356 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:05.118466 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:05.119408 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:05.119986 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:05.121198 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:05.121453 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:05.121579 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:05.122612 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:05.122616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:05.122619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:49:05.122636 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:05.122658 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:49:05.122669 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:05.122756 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:05.122986 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 17:49:05.123052 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:49:05.124292 28874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:05.115824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34529 (local address 127.13.38.2:42701)
0812 17:49:05.116584 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:05.116627 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:05.118026 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:05.118651 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:05.118658 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:05.118742 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:05.119272 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:05.119285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:05.120145 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:05.120148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:05.121081 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:05.121092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:05.121635 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:05.121639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:05.121783 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:05.122342 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:49:05.122343 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:05.122382 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:49:05.122446 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:49:05.122800 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:49:05.122810 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:49:05.122818 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 17:49:05.122965 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 17:49:05.123211 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":119,"threads_started":1}
I20260812 17:49:05.125406 28867 heartbeater.cc:347] Connected to a master server at 127.13.38.62:34529
I20260812 17:49:05.125592 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34529
--builtin_ntp_servers=127.13.38.20:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 17:49:05.125677 28867 heartbeater.cc:464] Registering TS with master...
I20260812 17:49:05.126217 28867 heartbeater.cc:511] Master 127.13.38.62:34529 requested a full tablet report, sending...
I20260812 17:49:05.127305 28354 ts_manager.cc:194] Registered new tserver with Master: 9c3fe4604c3c4d84b98c60f629b6be4d (127.13.38.2:38023)
I20260812 17:49:05.128165 28354 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.38.2:42701
W20260812 17:49:05.247949 28875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:05.248256 28875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:05.248296 28875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:05.253562 28875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:05.253651 28875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:05.253759 28875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:49:05.258872 28875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:05.260169 28875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:05.261668 28875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:05.271870 28883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:05.271914 28888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:05.271872 28886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:05.272049 28875 server_base.cc:1034] running on GCE node
I20260812 17:49:05.272514 28875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:05.273285 28875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:05.274474 28875 hybrid_clock.cc:648] HybridClock initialized: now 1786556945274443 us; error 58 us; skew 500 ppm
I20260812 17:49:05.276520 28875 webserver.cc:460] Webserver started at http://127.13.38.3:36367/ using document root <none> and password file <none>
I20260812 17:49:05.277163 28875 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:05.277279 28875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:05.277539 28875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:49:05.279865 28875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "bac46821c4fb46a4bf18d441878bbb4c"
format_stamp: "Formatted at 2026-08-12 17:49:05 on dist-test-slave-sb2z"
server_key: "b39765abf5b66c62bc46e7671c22fdef"
server_key_iv: "84db3577e4367cfd3c34840da42a3630"
server_key_version: "encryptionkey@0"
I20260812 17:49:05.280525 28875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "bac46821c4fb46a4bf18d441878bbb4c"
format_stamp: "Formatted at 2026-08-12 17:49:05 on dist-test-slave-sb2z"
server_key: "b39765abf5b66c62bc46e7671c22fdef"
server_key_iv: "84db3577e4367cfd3c34840da42a3630"
server_key_version: "encryptionkey@0"
I20260812 17:49:05.284778 28875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:49:05.287719 28897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:05.289222 28875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:49:05.289433 28875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "bac46821c4fb46a4bf18d441878bbb4c"
format_stamp: "Formatted at 2026-08-12 17:49:05 on dist-test-slave-sb2z"
server_key: "b39765abf5b66c62bc46e7671c22fdef"
server_key_iv: "84db3577e4367cfd3c34840da42a3630"
server_key_version: "encryptionkey@0"
I20260812 17:49:05.289620 28875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:05.313472 28875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:05.314321 28875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:05.314551 28875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:05.315236 28875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:49:05.316454 28875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:49:05.316530 28875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:05.316613 28875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:49:05.316651 28875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:05.334674 28875 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:36361
I20260812 17:49:05.334702 29083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:36361 every 8 connection(s)
I20260812 17:49:05.336169 28875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:49:05.345887 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 28875
I20260812 17:49:05.346064 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:49:05.346473 13464 external_mini_cluster.cc:1428] Setting key 99bd4f81df9c4648966ccd4d3608d7c5
I20260812 17:49:05.349102 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:05.338324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:51353 (local address 127.13.38.62:34529)
0812 17:49:05.338485 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:05.338490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:05.340399 (+ 1909us) server_negotiation.cc:407] Connection header received
0812 17:49:05.342565 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:05.342570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:05.342643 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:05.342758 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:05.344182 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:05.344903 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:05.346409 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:05.346729 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:05.346852 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:05.348474 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:05.348478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:05.348481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:49:05.348494 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:05.348516 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:49:05.348525 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:05.348570 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:05.348840 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 17:49:05.348921 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 17:49:05.350416 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:05.338937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34529 (local address 127.13.38.3:51353)
0812 17:49:05.340202 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:05.340274 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:05.342299 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:05.343031 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:05.343061 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:05.343202 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:05.344001 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:05.344021 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:05.345055 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:05.345060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:05.346277 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:05.346287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:05.347194 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:05.347201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:05.347372 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:05.348142 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:49:05.348144 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:05.348188 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:49:05.348299 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:49:05.348649 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:49:05.348658 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:49:05.348664 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 17:49:05.348783 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 17:49:05.349119 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":171,"threads_started":1}
I20260812 17:49:05.351791 29084 heartbeater.cc:347] Connected to a master server at 127.13.38.62:34529
I20260812 17:49:05.352136 29084 heartbeater.cc:464] Registering TS with master...
I20260812 17:49:05.352962 29084 heartbeater.cc:511] Master 127.13.38.62:34529 requested a full tablet report, sending...
I20260812 17:49:05.354326 28354 ts_manager.cc:194] Registered new tserver with Master: bac46821c4fb46a4bf18d441878bbb4c (127.13.38.3:36361)
I20260812 17:49:05.355003 28354 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.38.3:51353
I20260812 17:49:05.362504 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 17:49:05.366106 28429 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:05.364969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52212 (local address 127.13.38.62:34529)
0812 17:49:05.365111 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:05.365116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:05.365376 (+ 260us) server_negotiation.cc:407] Connection header received
0812 17:49:05.365519 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:05.365522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:05.365579 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:05.365648 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:05.365946 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52212: BlockingRecv error: recv got EOF from 127.0.0.1:52212 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 17:49:05.366241 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 28460
I20260812 17:49:05.392997 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 28668
I20260812 17:49:05.405946 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 28875
I20260812 17:49:05.420830 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 28306
2026-08-12T17:49:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1052 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T17:49:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:49:05Z Disabled control of system clock
I20260812 17:49:05.459151 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:36499
--webserver_interface=127.13.38.62
--webserver_port=0
--builtin_ntp_servers=127.13.38.20:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:36499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 17:49:05.578532 29101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:05.578843 29101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:05.578884 29101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:05.583140 29101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:49:05.583220 29101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:05.583235 29101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:05.583252 29101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:49:05.583268 29101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:49:05.588342 29101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:36499
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:36499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:05.589555 29101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:05.591012 29101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:05.599933 29113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:05.599854 29112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:05.600031 29101 server_base.cc:1034] running on GCE node
W20260812 17:49:05.599874 29117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:05.600827 29101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:05.602034 29101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:05.603207 29101 hybrid_clock.cc:648] HybridClock initialized: now 1786556945603177 us; error 51 us; skew 500 ppm
I20260812 17:49:05.605381 29101 webserver.cc:460] Webserver started at http://127.13.38.62:38709/ using document root <none> and password file <none>
I20260812 17:49:05.605988 29101 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:05.606087 29101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:05.606338 29101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:49:05.608600 29101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "1b481a0e79e740c6afcf12638c9b8f5d"
format_stamp: "Formatted at 2026-08-12 17:49:05 on dist-test-slave-sb2z"
server_key: "00ab400625c9feef482b3cf44c8f6a35"
server_key_iv: "61ddb1ffe06941bcd69ab1573ba2eb43"
server_key_version: "encryptionkey@0"
I20260812 17:49:05.609289 29101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "1b481a0e79e740c6afcf12638c9b8f5d"
format_stamp: "Formatted at 2026-08-12 17:49:05 on dist-test-slave-sb2z"
server_key: "00ab400625c9feef482b3cf44c8f6a35"
server_key_iv: "61ddb1ffe06941bcd69ab1573ba2eb43"
server_key_version: "encryptionkey@0"
I20260812 17:49:05.613407 29101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:49:05.616114 29128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:05.617453 29101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:49:05.617605 29101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "1b481a0e79e740c6afcf12638c9b8f5d"
format_stamp: "Formatted at 2026-08-12 17:49:05 on dist-test-slave-sb2z"
server_key: "00ab400625c9feef482b3cf44c8f6a35"
server_key_iv: "61ddb1ffe06941bcd69ab1573ba2eb43"
server_key_version: "encryptionkey@0"
I20260812 17:49:05.617731 29101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:05.622620 29101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:05.623256 29101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:05.623446 29101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:05.630729 29101 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:36499
I20260812 17:49:05.630733 29212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:36499 every 8 connection(s)
I20260812 17:49:05.631923 29101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:49:05.635879 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 29101
I20260812 17:49:05.636016 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:49:05.636276 13464 external_mini_cluster.cc:1428] Setting key 2a816a2c0fe3d4c5620116de66a5401f
I20260812 17:49:05.637154 29214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:49:05.643021 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:05.638034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.13.38.62:36499)
0812 17:49:05.638788 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:05.638805 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:05.638844 (+ 39us) server_negotiation.cc:407] Connection header received
0812 17:49:05.639674 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:05.639697 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:05.640083 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:05.640449 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:05.641387 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:05.641393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:05.641405 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:49:05.641417 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:05.641476 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:49:05.641493 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:05.641911 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:05.641993 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 17:49:05.642252 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":161,"threads_started":1}
I20260812 17:49:05.644491 29214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d: Bootstrap starting.
I20260812 17:49:05.646873 29214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d: Neither blocks nor log segments found. Creating new log.
I20260812 17:49:05.647790 29214 log.cc:824] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d: Log is configured to *not* fsync() on all Append() calls
I20260812 17:49:05.650014 29214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d: No bootstrap required, opened a new log
I20260812 17:49:05.653971 29214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b481a0e79e740c6afcf12638c9b8f5d" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 36499 } }
I20260812 17:49:05.654312 29214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:49:05.654381 29214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b481a0e79e740c6afcf12638c9b8f5d, State: Initialized, Role: FOLLOWER
I20260812 17:49:05.654889 29214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b481a0e79e740c6afcf12638c9b8f5d" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 36499 } }
I20260812 17:49:05.655025 29214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:49:05.655118 29214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:49:05.655225 29214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:49:05.656277 29214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b481a0e79e740c6afcf12638c9b8f5d" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 36499 } }
I20260812 17:49:05.656646 29214 leader_election.cc:302] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b481a0e79e740c6afcf12638c9b8f5d; no voters:
I20260812 17:49:05.656993 29214 leader_election.cc:288] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:49:05.657157 29224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:49:05.657407 29224 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [term 1 LEADER]: Becoming Leader. State: Replica: 1b481a0e79e740c6afcf12638c9b8f5d, State: Running, Role: LEADER
I20260812 17:49:05.657742 29224 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b481a0e79e740c6afcf12638c9b8f5d" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 36499 } }
I20260812 17:49:05.658298 29214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:49:05.659682 29226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b481a0e79e740c6afcf12638c9b8f5d. Latest consensus state: current_term: 1 leader_uuid: "1b481a0e79e740c6afcf12638c9b8f5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b481a0e79e740c6afcf12638c9b8f5d" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 36499 } } }
I20260812 17:49:05.659830 29226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [sys.catalog]: This master's current role is: LEADER
I20260812 17:49:05.659660 29225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b481a0e79e740c6afcf12638c9b8f5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b481a0e79e740c6afcf12638c9b8f5d" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 36499 } } }
I20260812 17:49:05.660095 29225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d [sys.catalog]: This master's current role is: LEADER
I20260812 17:49:05.660455 29234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:49:05.663374 29234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:49:05.669431 29234 catalog_manager.cc:1347] Generated new cluster ID: b9407361170e424297ae523514d4606b
I20260812 17:49:05.669523 29234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:49:05.679013 29234 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:49:05.679397 29234 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:49:05.687155 29234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b481a0e79e740c6afcf12638c9b8f5d: Generated new TSK 0
I20260812 17:49:05.688174 29234 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:49:05.697176 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:36499
--builtin_ntp_servers=127.13.38.20:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 17:49:05.826841 29250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:05.827174 29250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:05.827217 29250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:05.831581 29250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:05.831660 29250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:05.831722 29250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:49:05.836798 29250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:36499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:05.837944 29250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:05.839363 29250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:05.851032 29257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:05.851176 29266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:05.851127 29250 server_base.cc:1034] running on GCE node
W20260812 17:49:05.851145 29260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:05.851866 29250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:05.852567 29250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:05.853751 29250 hybrid_clock.cc:648] HybridClock initialized: now 1786556945853716 us; error 51 us; skew 500 ppm
I20260812 17:49:05.856037 29250 webserver.cc:460] Webserver started at http://127.13.38.1:34731/ using document root <none> and password file <none>
I20260812 17:49:05.856691 29250 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:05.856761 29250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:05.857030 29250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:49:05.859427 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "bdc0d6db063f4588abcf72b7d13b0b6e"
format_stamp: "Formatted at 2026-08-12 17:49:05 on dist-test-slave-sb2z"
server_key: "cb082dc4a04c815c6b5fae837b4a744a"
server_key_iv: "3e71dfe71ccce05c0848f5d23b8d3b59"
server_key_version: "encryptionkey@0"
I20260812 17:49:05.860244 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "bdc0d6db063f4588abcf72b7d13b0b6e"
format_stamp: "Formatted at 2026-08-12 17:49:05 on dist-test-slave-sb2z"
server_key: "cb082dc4a04c815c6b5fae837b4a744a"
server_key_iv: "3e71dfe71ccce05c0848f5d23b8d3b59"
server_key_version: "encryptionkey@0"
I20260812 17:49:05.864495 29250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:49:05.867617 29276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:05.869208 29250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:49:05.869411 29250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "bdc0d6db063f4588abcf72b7d13b0b6e"
format_stamp: "Formatted at 2026-08-12 17:49:05 on dist-test-slave-sb2z"
server_key: "cb082dc4a04c815c6b5fae837b4a744a"
server_key_iv: "3e71dfe71ccce05c0848f5d23b8d3b59"
server_key_version: "encryptionkey@0"
I20260812 17:49:05.869575 29250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:05.874938 29250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:05.875735 29250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:05.875938 29250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:05.876557 29250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:49:05.877579 29250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:49:05.877653 29250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:05.877731 29250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:49:05.877775 29250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:05.887070 29456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:44729 every 8 connection(s)
I20260812 17:49:05.887131 29250 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:44729
I20260812 17:49:05.888391 29250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:49:05.891867 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 29250
I20260812 17:49:05.892038 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:49:05.892393 13464 external_mini_cluster.cc:1428] Setting key e12207ee8a66ab76417584a951605e60
I20260812 17:49:05.894817 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:36499
--builtin_ntp_servers=127.13.38.20:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 17:49:05.896248 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:05.890724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:33971 (local address 127.13.38.62:36499)
0812 17:49:05.890976 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:05.890981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:05.892282 (+ 1301us) server_negotiation.cc:407] Connection header received
0812 17:49:05.894094 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:05.894097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:05.894159 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:05.894248 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:05.895616 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:05.895623 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:05.895629 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:49:05.895647 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:05.895676 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:49:05.895690 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:05.895794 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:05.895997 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 17:49:05.896056 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260812 17:49:05.897512 29461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:05.891146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:36499 (local address 127.13.38.1:33971)
0812 17:49:05.892069 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:05.892137 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:05.893811 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:05.894422 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:05.894429 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:05.894533 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:05.895267 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:49:05.895269 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:05.895315 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:49:05.895391 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:49:05.895826 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:49:05.895836 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:49:05.895842 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 17:49:05.896046 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 17:49:05.896287 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":141,"threads_started":1}
I20260812 17:49:05.898919 29459 heartbeater.cc:347] Connected to a master server at 127.13.38.62:36499
I20260812 17:49:05.899194 29459 heartbeater.cc:464] Registering TS with master...
I20260812 17:49:05.899763 29459 heartbeater.cc:511] Master 127.13.38.62:36499 requested a full tablet report, sending...
I20260812 17:49:05.901564 29150 ts_manager.cc:194] Registered new tserver with Master: bdc0d6db063f4588abcf72b7d13b0b6e (127.13.38.1:44729)
W20260812 17:49:06.017304 29462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:06.017612 29462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:06.017655 29462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:06.022187 29462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:06.022274 29462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:06.022367 29462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:49:06.028362 29462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:36499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:06.029655 29462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:06.031186 29462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:06.042156 29471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:06.042196 29469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:06.042156 29468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:06.042418 29462 server_base.cc:1034] running on GCE node
I20260812 17:49:06.042980 29462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:06.043804 29462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:06.045015 29462 hybrid_clock.cc:648] HybridClock initialized: now 1786556946044994 us; error 35 us; skew 500 ppm
I20260812 17:49:06.047075 29462 webserver.cc:460] Webserver started at http://127.13.38.2:33625/ using document root <none> and password file <none>
I20260812 17:49:06.047773 29462 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:06.047873 29462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:06.048137 29462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:49:06.050442 29462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "927e75ad1e664f95976cf32c54f9ed7b"
format_stamp: "Formatted at 2026-08-12 17:49:06 on dist-test-slave-sb2z"
server_key: "96caec7757760d8fb2901d34901aa104"
server_key_iv: "42e6842843b326380e731c7e0ce5a460"
server_key_version: "encryptionkey@0"
I20260812 17:49:06.051174 29462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "927e75ad1e664f95976cf32c54f9ed7b"
format_stamp: "Formatted at 2026-08-12 17:49:06 on dist-test-slave-sb2z"
server_key: "96caec7757760d8fb2901d34901aa104"
server_key_iv: "42e6842843b326380e731c7e0ce5a460"
server_key_version: "encryptionkey@0"
I20260812 17:49:06.055277 29462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 17:49:06.058353 29478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:06.059926 29462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:49:06.060101 29462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "927e75ad1e664f95976cf32c54f9ed7b"
format_stamp: "Formatted at 2026-08-12 17:49:06 on dist-test-slave-sb2z"
server_key: "96caec7757760d8fb2901d34901aa104"
server_key_iv: "42e6842843b326380e731c7e0ce5a460"
server_key_version: "encryptionkey@0"
I20260812 17:49:06.060241 29462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:06.065153 29462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:06.065804 29462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:06.066017 29462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:06.066596 29462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:49:06.067615 29462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:49:06.067683 29462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:06.067754 29462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:49:06.067806 29462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:06.077458 29462 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:37665
I20260812 17:49:06.077510 29661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:37665 every 8 connection(s)
I20260812 17:49:06.078500 29462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:49:06.084012 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 29462
I20260812 17:49:06.084213 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:49:06.084566 13464 external_mini_cluster.cc:1428] Setting key bce0c65d7d5c27a598ba371eba308b2e
I20260812 17:49:06.085536 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:06.080696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:60103 (local address 127.13.38.62:36499)
0812 17:49:06.080865 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:06.080869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:06.082097 (+ 1228us) server_negotiation.cc:407] Connection header received
0812 17:49:06.083499 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:06.083503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:06.083590 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:06.083705 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:06.085040 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:06.085043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:06.085045 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:49:06.085058 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:06.085079 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:49:06.085087 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:06.085194 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:06.085347 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 17:49:06.085394 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 17:49:06.086551 29666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:06.081136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:36499 (local address 127.13.38.2:60103)
0812 17:49:06.081902 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:06.081948 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:06.083268 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:06.083904 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:06.083911 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:06.084005 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:06.084740 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:49:06.084743 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:06.084796 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:49:06.084869 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:49:06.085175 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:49:06.085185 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:49:06.085190 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 17:49:06.085385 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 17:49:06.085672 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":121,"threads_started":1}
I20260812 17:49:06.086583 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:36499
--builtin_ntp_servers=127.13.38.20:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 17:49:06.087857 29662 heartbeater.cc:347] Connected to a master server at 127.13.38.62:36499
I20260812 17:49:06.088141 29662 heartbeater.cc:464] Registering TS with master...
I20260812 17:49:06.088683 29662 heartbeater.cc:511] Master 127.13.38.62:36499 requested a full tablet report, sending...
I20260812 17:49:06.089931 29150 ts_manager.cc:194] Registered new tserver with Master: 927e75ad1e664f95976cf32c54f9ed7b (127.13.38.2:37665)
W20260812 17:49:06.207346 29668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:06.207665 29668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:06.207710 29668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:06.211710 29668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:06.211771 29668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:06.211828 29668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:49:06.216831 29668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:36499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:06.217947 29668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:06.219354 29668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:06.229909 29675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:06.230010 29676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:06.229909 29678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:06.230192 29668 server_base.cc:1034] running on GCE node
I20260812 17:49:06.230742 29668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:06.231489 29668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:06.232761 29668 hybrid_clock.cc:648] HybridClock initialized: now 1786556946232722 us; error 50 us; skew 500 ppm
I20260812 17:49:06.235248 29668 webserver.cc:460] Webserver started at http://127.13.38.3:38415/ using document root <none> and password file <none>
I20260812 17:49:06.235968 29668 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:06.236038 29668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:06.236279 29668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:49:06.238531 29668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "43a0c43fdba642e69b204bbc8efcacbd"
format_stamp: "Formatted at 2026-08-12 17:49:06 on dist-test-slave-sb2z"
server_key: "22b47590db53a921f08583e70851c16c"
server_key_iv: "96f3e888af9b01f62c44832aa2b4d859"
server_key_version: "encryptionkey@0"
I20260812 17:49:06.239164 29668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "43a0c43fdba642e69b204bbc8efcacbd"
format_stamp: "Formatted at 2026-08-12 17:49:06 on dist-test-slave-sb2z"
server_key: "22b47590db53a921f08583e70851c16c"
server_key_iv: "96f3e888af9b01f62c44832aa2b4d859"
server_key_version: "encryptionkey@0"
I20260812 17:49:06.243306 29668 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 17:49:06.246304 29689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:06.247815 29668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:49:06.247961 29668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "43a0c43fdba642e69b204bbc8efcacbd"
format_stamp: "Formatted at 2026-08-12 17:49:06 on dist-test-slave-sb2z"
server_key: "22b47590db53a921f08583e70851c16c"
server_key_iv: "96f3e888af9b01f62c44832aa2b4d859"
server_key_version: "encryptionkey@0"
I20260812 17:49:06.248091 29668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:06.252991 29668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:06.253633 29668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:06.253818 29668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:06.254446 29668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:49:06.255643 29668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:49:06.255716 29668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:06.255779 29668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:49:06.255813 29668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:06.265522 29668 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:43605
I20260812 17:49:06.265588 29875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:43605 every 8 connection(s)
I20260812 17:49:06.266502 29668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:49:06.272826 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:06.268430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:49319 (local address 127.13.38.62:36499)
0812 17:49:06.268587 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:06.268591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:06.269614 (+ 1023us) server_negotiation.cc:407] Connection header received
0812 17:49:06.270996 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:06.270999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:06.271048 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:06.271126 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:06.272308 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:06.272312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:06.272314 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:49:06.272334 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:06.272356 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:49:06.272365 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:06.272467 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:06.272626 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 17:49:06.272679 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 17:49:06.273895 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:06.268793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:36499 (local address 127.13.38.3:49319)
0812 17:49:06.269456 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:06.269496 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:06.270766 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:06.271291 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:06.271299 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:06.271389 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:06.272000 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:49:06.272030 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:06.272073 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:49:06.272145 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:49:06.272460 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:49:06.272469 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:49:06.272478 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 17:49:06.272674 (+ 196us) client_negotiation.cc:240] Negotiation successful
0812 17:49:06.272928 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
I20260812 17:49:06.275233 29876 heartbeater.cc:347] Connected to a master server at 127.13.38.62:36499
I20260812 17:49:06.275499 29876 heartbeater.cc:464] Registering TS with master...
I20260812 17:49:06.275985 29876 heartbeater.cc:511] Master 127.13.38.62:36499 requested a full tablet report, sending...
I20260812 17:49:06.276077 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 29668
I20260812 17:49:06.276232 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:49:06.276527 13464 external_mini_cluster.cc:1428] Setting key 089e5fbaf179830bdaafa9cd227beb46
I20260812 17:49:06.277202 29150 ts_manager.cc:194] Registered new tserver with Master: 43a0c43fdba642e69b204bbc8efcacbd (127.13.38.3:43605)
I20260812 17:49:06.278637 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 17:49:06.281884 29220 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:49:06.280774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34082 (local address 127.13.38.62:36499)
0812 17:49:06.280971 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:06.280975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:06.281142 (+ 167us) server_negotiation.cc:407] Connection header received
0812 17:49:06.281303 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:06.281306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:06.281345 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:06.281442 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:06.281700 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34082: BlockingRecv error: recv got EOF from 127.0.0.1:34082 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 17:49:06.282051 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 29250
I20260812 17:49:06.296375 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 29462
I20260812 17:49:06.317911 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 29668
I20260812 17:49:06.331493 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 29101
2026-08-12T17:49:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (913 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:49:06 dist-test-slave-sb2z krb5kdc[29894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:49:06 dist-test-slave-sb2z krb5kdc[29894](info): set up 2 sockets
Aug 12 17:49:06 dist-test-slave-sb2z krb5kdc[29894](info): commencing operation
krb5kdc: starting...
W20260812 17:49:07.664021 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:49:07 dist-test-slave-sb2z krb5kdc[29894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:49:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:49:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:49:07.808203 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:35887
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:35887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:49:07.930370 29912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:07.930683 29912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:07.930727 29912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:07.934968 29912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:49:07.935060 29912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:07.935079 29912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:07.935097 29912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:49:07.935113 29912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:49:07.941320 29912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:35887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:35887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:07.942691 29912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:07.943915 29912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:07.953791 29922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:07.953846 29923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:07.953796 29926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:07.954048 29912 server_base.cc:1034] running on GCE node
I20260812 17:49:07.954785 29912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:07.956130 29912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:07.957348 29912 hybrid_clock.cc:648] HybridClock initialized: now 1786556947957312 us; error 52 us; skew 500 ppm
Aug 12 17:49:07 dist-test-slave-sb2z krb5kdc[29894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556947, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:49:07.960588 29912 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:49:07.961714 29912 webserver.cc:460] Webserver started at http://127.13.38.62:38549/ using document root <none> and password file <none>
I20260812 17:49:07.962388 29912 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:07.962458 29912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:07.962709 29912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:49:07.965135 29912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "0d7dca8308eb4235a32e2338e4f12cec"
format_stamp: "Formatted at 2026-08-12 17:49:07 on dist-test-slave-sb2z"
server_key: "da5e534cd23333af7925ada098e5f460"
server_key_iv: "2fbd94ba62824ea180fd0bb83c5de7d6"
server_key_version: "encryptionkey@0"
I20260812 17:49:07.965814 29912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "0d7dca8308eb4235a32e2338e4f12cec"
format_stamp: "Formatted at 2026-08-12 17:49:07 on dist-test-slave-sb2z"
server_key: "da5e534cd23333af7925ada098e5f460"
server_key_iv: "2fbd94ba62824ea180fd0bb83c5de7d6"
server_key_version: "encryptionkey@0"
I20260812 17:49:07.970166 29912 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:49:07.973348 29948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:07.975067 29912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:49:07.975265 29912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "0d7dca8308eb4235a32e2338e4f12cec"
format_stamp: "Formatted at 2026-08-12 17:49:07 on dist-test-slave-sb2z"
server_key: "da5e534cd23333af7925ada098e5f460"
server_key_iv: "2fbd94ba62824ea180fd0bb83c5de7d6"
server_key_version: "encryptionkey@0"
I20260812 17:49:07.975423 29912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:07.997960 29912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:08.001662 29912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:08.001917 29912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:08.010592 29912 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:35887
I20260812 17:49:08.010991 30029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:35887 every 8 connection(s)
I20260812 17:49:08.011821 29912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:49:08.015131 30030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:49:08.017230 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 29912
I20260812 17:49:08.017376 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:49:08.017714 13464 external_mini_cluster.cc:1428] Setting key f0747966f8191985530f878ab2cfde4a
I20260812 17:49:08.022766 30030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec: Bootstrap starting.
I20260812 17:49:08.025476 30030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec: Neither blocks nor log segments found. Creating new log.
I20260812 17:49:08.026463 30030 log.cc:824] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec: Log is configured to *not* fsync() on all Append() calls
Aug 12 17:49:08 dist-test-slave-sb2z krb5kdc[29894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:49:08.028988 30030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec: No bootstrap required, opened a new log
I20260812 17:49:08.032783 30030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7dca8308eb4235a32e2338e4f12cec" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35887 } }
I20260812 17:49:08.033103 30030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:49:08.033190 30030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d7dca8308eb4235a32e2338e4f12cec, State: Initialized, Role: FOLLOWER
I20260812 17:49:08.033694 30030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7dca8308eb4235a32e2338e4f12cec" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35887 } }
I20260812 17:49:08.033828 30030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:49:08.033908 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.019462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46680 (local address 127.13.38.62:35887)
0812 17:49:08.020093 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:08.020111 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:08.020152 (+ 41us) server_negotiation.cc:407] Connection header received
0812 17:49:08.020812 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:08.020830 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:08.021203 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:08.021595 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:08.022492 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.023680 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.024586 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.025085 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.025267 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.028442 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:08.028449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:08.028468 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:49:08.028503 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:08.030602 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:08.031162 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:08.031168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:08.031173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:08.031297 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:08.031649 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:08.031653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:08.031655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:08.032095 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:08.032333 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:08.032643 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 17:49:08.032959 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":166,"threads_started":1}
I20260812 17:49:08.033917 30030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:49:08.034111 30030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:49:08.035223 30030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7dca8308eb4235a32e2338e4f12cec" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35887 } }
I20260812 17:49:08.035655 30030 leader_election.cc:302] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d7dca8308eb4235a32e2338e4f12cec; no voters:
I20260812 17:49:08.036083 30030 leader_election.cc:288] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:49:08.036296 30037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:49:08.036564 30037 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [term 1 LEADER]: Becoming Leader. State: Replica: 0d7dca8308eb4235a32e2338e4f12cec, State: Running, Role: LEADER
I20260812 17:49:08.036900 30037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7dca8308eb4235a32e2338e4f12cec" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35887 } }
I20260812 17:49:08.037479 30030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:49:08.038704 30039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d7dca8308eb4235a32e2338e4f12cec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7dca8308eb4235a32e2338e4f12cec" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35887 } } }
I20260812 17:49:08.038842 30039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [sys.catalog]: This master's current role is: LEADER
I20260812 17:49:08.039131 30041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d7dca8308eb4235a32e2338e4f12cec. Latest consensus state: current_term: 1 leader_uuid: "0d7dca8308eb4235a32e2338e4f12cec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7dca8308eb4235a32e2338e4f12cec" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35887 } } }
I20260812 17:49:08.039227 30041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec [sys.catalog]: This master's current role is: LEADER
I20260812 17:49:08.039364 30050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:49:08.042752 30050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:49:08.048744 30050 catalog_manager.cc:1347] Generated new cluster ID: 8d4b0250acd94279b52a68dd2697bcd6
I20260812 17:49:08.048837 30050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:49:08.057919 30050 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:49:08.059297 30050 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:49:08.067252 30050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec: Generated new TSK 0
I20260812 17:49:08.068173 30050 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:49:08.111150 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35887
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:49:08.233760 30073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:08.234117 30073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:08.234202 30073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:08.238818 30073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:08.238909 30073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:08.239007 30073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:49:08.245236 30073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:08.246523 30073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:08.247753 30073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:08.258195 30078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:08.258347 30073 server_base.cc:1034] running on GCE node
W20260812 17:49:08.258203 30079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:08.258245 30083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:08.259011 30073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:08.259796 30073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:08.261030 30073 hybrid_clock.cc:648] HybridClock initialized: now 1786556948261012 us; error 33 us; skew 500 ppm
Aug 12 17:49:08 dist-test-slave-sb2z krb5kdc[29894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556948, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:49:08.269094 30073 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:49:08.270275 30073 webserver.cc:460] Webserver started at http://127.13.38.1:36177/ using document root <none> and password file <none>
I20260812 17:49:08.270977 30073 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:08.271095 30073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:08.271365 30073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:49:08.273757 30073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "2ac4321512824119b23120814640f1ec"
format_stamp: "Formatted at 2026-08-12 17:49:08 on dist-test-slave-sb2z"
server_key: "c674b1b8cb70408b04fd436a93c1aae5"
server_key_iv: "4fdce831077a368d30c34d663a0f50e1"
server_key_version: "encryptionkey@0"
I20260812 17:49:08.274412 30073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "2ac4321512824119b23120814640f1ec"
format_stamp: "Formatted at 2026-08-12 17:49:08 on dist-test-slave-sb2z"
server_key: "c674b1b8cb70408b04fd436a93c1aae5"
server_key_iv: "4fdce831077a368d30c34d663a0f50e1"
server_key_version: "encryptionkey@0"
I20260812 17:49:08.278716 30073 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 17:49:08.282015 30097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:08.283511 30073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:49:08.283778 30073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "2ac4321512824119b23120814640f1ec"
format_stamp: "Formatted at 2026-08-12 17:49:08 on dist-test-slave-sb2z"
server_key: "c674b1b8cb70408b04fd436a93c1aae5"
server_key_iv: "4fdce831077a368d30c34d663a0f50e1"
server_key_version: "encryptionkey@0"
I20260812 17:49:08.283921 30073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:08.311820 30073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:08.315362 30073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:08.315636 30073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:08.316325 30073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:49:08.317355 30073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:49:08.317433 30073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:08.317504 30073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:49:08.317535 30073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:08.327368 30073 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:44795
I20260812 17:49:08.327394 30276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:44795 every 8 connection(s)
I20260812 17:49:08.328418 30073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:49:08.330647 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 30073
I20260812 17:49:08.330821 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:49:08.331200 13464 external_mini_cluster.cc:1428] Setting key ec5e9b92e15a6aa12ed76940b9eb80cf
Aug 12 17:49:08 dist-test-slave-sb2z krb5kdc[29894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556948, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:49:08.344992 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.330439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:46915 (local address 127.13.38.62:35887)
0812 17:49:08.330660 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:08.330668 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:08.331605 (+ 937us) server_negotiation.cc:407] Connection header received
0812 17:49:08.333313 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:08.333316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:08.333369 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:08.333481 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:08.334744 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.335294 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.336609 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.336845 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.336963 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.340592 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:08.340595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:08.340597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:49:08.340626 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:08.342260 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:08.343044 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:08.343047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:08.343048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:08.343229 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:08.343762 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:08.343765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:08.343766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:08.343968 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:08.344145 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:08.344737 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 17:49:08.344857 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 17:49:08.346015 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.330891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35887 (local address 127.13.38.1:46915)
0812 17:49:08.331413 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:08.331471 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:08.333090 (+ 1619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:08.333687 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:08.333694 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:08.334076 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:08.334573 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:08.334587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.335433 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.335436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:08.336470 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:08.336480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.337070 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.337078 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:08.337292 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.337946 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:08.337964 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:08.340362 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:49:08.342434 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:49:08.342445 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:49:08.342471 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:49:08.342888 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:49:08.343347 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:49:08.343353 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:49:08.343357 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:49:08.343655 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:49:08.344157 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:49:08.344163 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:49:08.344473 (+ 310us) client_negotiation.cc:769] Sending connection context
0812 17:49:08.344656 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 17:49:08.344976 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
I20260812 17:49:08.347433 30279 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35887
I20260812 17:49:08.347754 30279 heartbeater.cc:464] Registering TS with master...
I20260812 17:49:08.348320 30279 heartbeater.cc:511] Master 127.13.38.62:35887 requested a full tablet report, sending...
I20260812 17:49:08.349999 29961 ts_manager.cc:194] Registered new tserver with Master: 2ac4321512824119b23120814640f1ec (127.13.38.1:44795)
I20260812 17:49:08.351292 29961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:46915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:49:08.366585 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35887
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:49:08.485810 30288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:08.486177 30288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:08.486277 30288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:08.490741 30288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:08.490854 30288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:08.490954 30288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:49:08.496671 30288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:08.497874 30288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:08.498998 30288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:08.509433 30297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:08.509415 30302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:08.509421 30298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:08.509696 30288 server_base.cc:1034] running on GCE node
I20260812 17:49:08.510196 30288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:08.511107 30288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:08.512331 30288 hybrid_clock.cc:648] HybridClock initialized: now 1786556948512280 us; error 86 us; skew 500 ppm
Aug 12 17:49:08 dist-test-slave-sb2z krb5kdc[29894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556948, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:49:08.515606 30288 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:49:08.516695 30288 webserver.cc:460] Webserver started at http://127.13.38.2:40069/ using document root <none> and password file <none>
I20260812 17:49:08.517345 30288 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:08.517452 30288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:08.517724 30288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:49:08.520160 30288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "cd2e91d9dbe74e16ade0422e4fe387fa"
format_stamp: "Formatted at 2026-08-12 17:49:08 on dist-test-slave-sb2z"
server_key: "5d4fa9b173a874af8197b6fccef562a6"
server_key_iv: "46415ea53a1cf4c0b51f7fb14b780fe0"
server_key_version: "encryptionkey@0"
I20260812 17:49:08.520825 30288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "cd2e91d9dbe74e16ade0422e4fe387fa"
format_stamp: "Formatted at 2026-08-12 17:49:08 on dist-test-slave-sb2z"
server_key: "5d4fa9b173a874af8197b6fccef562a6"
server_key_iv: "46415ea53a1cf4c0b51f7fb14b780fe0"
server_key_version: "encryptionkey@0"
I20260812 17:49:08.525220 30288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:49:08.528144 30329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:08.529570 30288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:49:08.529738 30288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "cd2e91d9dbe74e16ade0422e4fe387fa"
format_stamp: "Formatted at 2026-08-12 17:49:08 on dist-test-slave-sb2z"
server_key: "5d4fa9b173a874af8197b6fccef562a6"
server_key_iv: "46415ea53a1cf4c0b51f7fb14b780fe0"
server_key_version: "encryptionkey@0"
I20260812 17:49:08.529877 30288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:08.547106 30288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:08.550631 30288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:08.550863 30288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:08.551601 30288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:49:08.552570 30288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:49:08.552649 30288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:08.552726 30288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:49:08.552780 30288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:08.562857 30288 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:45567
I20260812 17:49:08.562911 30522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:45567 every 8 connection(s)
I20260812 17:49:08.563966 30288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:49:08.564811 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 30288
I20260812 17:49:08.564960 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:49:08.565351 13464 external_mini_cluster.cc:1428] Setting key 7765839b59825e85abbd9cd6e4df488c
Aug 12 17:49:08 dist-test-slave-sb2z krb5kdc[29894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556948, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
I20260812 17:49:08.581673 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.566168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:42211 (local address 127.13.38.62:35887)
0812 17:49:08.566306 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:08.566310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:08.567514 (+ 1204us) server_negotiation.cc:407] Connection header received
0812 17:49:08.569458 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:08.569461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:08.569509 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:08.569587 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:08.571246 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.571852 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.573962 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.574192 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.574278 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.577513 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:08.577516 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:08.577518 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:49:08.577546 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:08.579104 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:08.579848 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:08.579852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:08.579854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:08.579937 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:08.580323 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:08.580327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:08.580328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:08.580535 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:08.580699 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:08.581285 (+ 586us) server_negotiation.cc:299] Negotiation successful
0812 17:49:08.581484 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 17:49:08.582732 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.566727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35887 (local address 127.13.38.2:42211)
0812 17:49:08.567346 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:08.567410 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:08.568943 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:08.569793 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:08.569803 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:08.570403 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:08.571087 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:08.571103 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.571990 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.571998 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:08.573819 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:08.573832 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.574386 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.574392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:08.574680 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.575420 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:08.575452 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:08.577308 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:49:08.579268 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:49:08.579277 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:49:08.579295 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:49:08.579723 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:49:08.580039 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:49:08.580042 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:49:08.580044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:49:08.580213 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:49:08.580658 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:49:08.580666 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:49:08.581008 (+ 342us) client_negotiation.cc:769] Sending connection context
0812 17:49:08.581223 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 17:49:08.581600 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":181,"threads_started":1}
I20260812 17:49:08.584446 30523 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35887
I20260812 17:49:08.584770 30523 heartbeater.cc:464] Registering TS with master...
I20260812 17:49:08.585408 30523 heartbeater.cc:511] Master 127.13.38.62:35887 requested a full tablet report, sending...
I20260812 17:49:08.586711 29961 ts_manager.cc:194] Registered new tserver with Master: cd2e91d9dbe74e16ade0422e4fe387fa (127.13.38.2:45567)
I20260812 17:49:08.587489 29961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:42211
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:49:08.603227 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35887
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:49:08.724857 30529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:08.725168 30529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:08.725215 30529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:08.729471 30529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:08.729562 30529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:08.729628 30529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:49:08.735563 30529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:08.736739 30529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:08.737905 30529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:08.748472 30540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:08.748446 30537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:08.748519 30529 server_base.cc:1034] running on GCE node
W20260812 17:49:08.748445 30536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:08.749197 30529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:08.749922 30529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:08.751132 30529 hybrid_clock.cc:648] HybridClock initialized: now 1786556948751118 us; error 32 us; skew 500 ppm
Aug 12 17:49:08 dist-test-slave-sb2z krb5kdc[29894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556948, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:49:08.754148 30529 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:49:08.755190 30529 webserver.cc:460] Webserver started at http://127.13.38.3:45825/ using document root <none> and password file <none>
I20260812 17:49:08.755860 30529 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:08.755955 30529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:08.756206 30529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:49:08.758323 30529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "0b5b42ed8f414fd483555fb7fb775b1c"
format_stamp: "Formatted at 2026-08-12 17:49:08 on dist-test-slave-sb2z"
server_key: "326877719a6216f314eacacebce275e3"
server_key_iv: "0fdc76ce364b04179040e34d77722dd9"
server_key_version: "encryptionkey@0"
I20260812 17:49:08.758914 30529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "0b5b42ed8f414fd483555fb7fb775b1c"
format_stamp: "Formatted at 2026-08-12 17:49:08 on dist-test-slave-sb2z"
server_key: "326877719a6216f314eacacebce275e3"
server_key_iv: "0fdc76ce364b04179040e34d77722dd9"
server_key_version: "encryptionkey@0"
I20260812 17:49:08.762940 30529 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:49:08.765645 30549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:08.766934 30529 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 17:49:08.767081 30529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "0b5b42ed8f414fd483555fb7fb775b1c"
format_stamp: "Formatted at 2026-08-12 17:49:08 on dist-test-slave-sb2z"
server_key: "326877719a6216f314eacacebce275e3"
server_key_iv: "0fdc76ce364b04179040e34d77722dd9"
server_key_version: "encryptionkey@0"
I20260812 17:49:08.767208 30529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:08.801121 30529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:08.804831 30529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:08.805061 30529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:08.805737 30529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:49:08.806640 30529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:49:08.806713 30529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:08.806790 30529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:49:08.806843 30529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:08.816687 30529 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:33651
I20260812 17:49:08.816713 30727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:33651 every 8 connection(s)
I20260812 17:49:08.817641 30529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:49:08.822359 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 30529
I20260812 17:49:08.822525 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:49:08.822816 13464 external_mini_cluster.cc:1428] Setting key 18425d5bb0483cd93ec0e0e496c85fc9
Aug 12 17:49:08 dist-test-slave-sb2z krb5kdc[29894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556948, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:49:08.833444 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.819736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:54055 (local address 127.13.38.62:35887)
0812 17:49:08.819891 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:08.819895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:08.820791 (+ 896us) server_negotiation.cc:407] Connection header received
0812 17:49:08.822358 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:08.822362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:08.822425 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:08.822540 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:08.824141 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.824861 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.826177 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.826443 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.826550 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.829300 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:08.829303 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:08.829305 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:49:08.829333 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:08.831136 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:08.831817 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:08.831821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:08.831822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:08.831915 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:08.832302 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:08.832306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:08.832307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:08.832523 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:08.832639 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:08.833198 (+ 559us) server_negotiation.cc:299] Negotiation successful
0812 17:49:08.833313 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 17:49:08.834395 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.820081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35887 (local address 127.13.38.3:54055)
0812 17:49:08.820609 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:08.820656 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:08.822112 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:08.822731 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:08.822739 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:08.823196 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:08.823990 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:08.824011 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.825098 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.825102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:08.826036 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:08.826047 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.826619 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.826622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:08.826782 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.827441 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:08.827463 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:08.829099 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:49:08.831269 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:49:08.831274 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:49:08.831290 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:49:08.831696 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:49:08.832012 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:49:08.832015 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:49:08.832016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:49:08.832167 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:49:08.832651 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:49:08.832657 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:49:08.832921 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 17:49:08.833124 (+ 203us) client_negotiation.cc:240] Negotiation successful
0812 17:49:08.833413 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":126,"threads_started":1}
I20260812 17:49:08.836086 30730 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35887
I20260812 17:49:08.836390 30730 heartbeater.cc:464] Registering TS with master...
I20260812 17:49:08.837039 30730 heartbeater.cc:511] Master 127.13.38.62:35887 requested a full tablet report, sending...
I20260812 17:49:08.838280 29961 ts_manager.cc:194] Registered new tserver with Master: 0b5b42ed8f414fd483555fb7fb775b1c (127.13.38.3:33651)
I20260812 17:49:08.838913 29961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:54055
I20260812 17:49:08.849846 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:49:08.859899 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.852140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46692 (local address 127.13.38.62:35887)
0812 17:49:08.852282 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:08.852285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:08.852470 (+ 185us) server_negotiation.cc:407] Connection header received
0812 17:49:08.852563 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:08.852566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:08.852617 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:08.852694 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:08.853468 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.854018 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.854894 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.855224 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.855306 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.856791 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:08.856795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:08.856797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:49:08.856821 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:08.858148 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:08.858534 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:08.858537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:08.858538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:08.858613 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:08.858879 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:08.858882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:08.858883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:08.859166 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:08.859283 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:08.859663 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 17:49:08.859773 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:49:08.863193 29961 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:49:08.866400 29961 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:49:08.882617 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.876818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.13.38.1:44795)
0812 17:49:08.877231 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:08.877240 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:08.877270 (+ 30us) server_negotiation.cc:407] Connection header received
0812 17:49:08.877354 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:08.877359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:08.877589 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:08.877765 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:49:08.878077 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.879335 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.880986 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.881836 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.881966 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.882151 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:08.882326 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 17:49:08.882435 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":103,"threads_started":1}
I20260812 17:49:08.882596 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.875950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:44795 (local address 127.0.0.1:35436)
0812 17:49:08.876732 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:08.876747 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:08.876868 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:08.877769 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:08.877775 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:08.877801 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:49:08.877938 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:08.877945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.879492 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.879498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:08.880830 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:08.880841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.881983 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.881987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:08.882106 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.882145 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 17:49:08.882266 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 17:49:08.882412 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":91,"spinlock_wait_cycles":5248,"thread_start_us":113,"threads_started":1}
I20260812 17:49:08.882596 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.875911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:45567 (local address 127.0.0.1:52166)
0812 17:49:08.876663 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:08.876719 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:08.877382 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:08.878136 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:08.878139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:08.878157 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:49:08.878242 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:08.878248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.879422 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.879426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:08.881154 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:08.881165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.882215 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.882222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:08.882309 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.882330 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 17:49:08.882393 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 17:49:08.882447 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":63,"spinlock_wait_cycles":5632,"thread_start_us":132,"threads_started":1}
I20260812 17:49:08.883210 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.876980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57472 (local address 127.13.38.3:33651)
0812 17:49:08.877381 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:08.877392 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:08.877410 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:49:08.877474 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:08.877478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:08.877619 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:08.877765 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:49:08.878220 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.879376 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.881216 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.882008 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.882109 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.882260 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:08.882709 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 17:49:08.883053 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":124,"threads_started":1}
I20260812 17:49:08.883525 30752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.876044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52166 (local address 127.13.38.2:45567)
0812 17:49:08.877661 (+ 1617us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:08.877698 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:08.877726 (+ 28us) server_negotiation.cc:407] Connection header received
0812 17:49:08.877798 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:08.877803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:08.877983 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:08.878159 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:49:08.878346 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.879290 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.881254 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.881858 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.881991 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.882149 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:08.882668 (+ 519us) server_negotiation.cc:299] Negotiation successful
0812 17:49:08.883375 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1508,"thread_start_us":188,"threads_started":1}
I20260812 17:49:08.883657 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:08.876269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:33651 (local address 127.0.0.1:57472)
0812 17:49:08.877102 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:08.877116 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:08.877192 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:08.877924 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:08.877927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:08.877955 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:49:08.878078 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:08.878085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.879482 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.879484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:08.881077 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:08.881089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:08.882331 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:08.882336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:08.882433 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:08.882456 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 17:49:08.882541 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 17:49:08.882615 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":121,"thread_start_us":106,"threads_started":1}
I20260812 17:49:08.886193 30411 tablet_service.cc:1467] Processing CreateTablet for tablet 73049317cf7244e7acaa237af33346a4 (DEFAULT_TABLE table=test-table [id=783ea851b8274b8e822b70f69f565d0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:49:08.886219 30170 tablet_service.cc:1467] Processing CreateTablet for tablet 73049317cf7244e7acaa237af33346a4 (DEFAULT_TABLE table=test-table [id=783ea851b8274b8e822b70f69f565d0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:49:08.886478 30624 tablet_service.cc:1467] Processing CreateTablet for tablet 73049317cf7244e7acaa237af33346a4 (DEFAULT_TABLE table=test-table [id=783ea851b8274b8e822b70f69f565d0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:49:08.887398 30411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73049317cf7244e7acaa237af33346a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:49:08.887408 30624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73049317cf7244e7acaa237af33346a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:49:08.887406 30170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73049317cf7244e7acaa237af33346a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:49:08.894529 30756 tablet_bootstrap.cc:492] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa: Bootstrap starting.
I20260812 17:49:08.895917 30757 tablet_bootstrap.cc:492] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec: Bootstrap starting.
I20260812 17:49:08.897554 30756 tablet_bootstrap.cc:654] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa: Neither blocks nor log segments found. Creating new log.
I20260812 17:49:08.897992 30757 tablet_bootstrap.cc:654] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec: Neither blocks nor log segments found. Creating new log.
I20260812 17:49:08.898566 30756 log.cc:824] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa: Log is configured to *not* fsync() on all Append() calls
I20260812 17:49:08.898834 30757 log.cc:824] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec: Log is configured to *not* fsync() on all Append() calls
I20260812 17:49:08.899554 30758 tablet_bootstrap.cc:492] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c: Bootstrap starting.
I20260812 17:49:08.900818 30756 tablet_bootstrap.cc:492] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa: No bootstrap required, opened a new log
I20260812 17:49:08.901088 30756 ts_tablet_manager.cc:1397] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260812 17:49:08.901648 30758 tablet_bootstrap.cc:654] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c: Neither blocks nor log segments found. Creating new log.
I20260812 17:49:08.902807 30758 log.cc:824] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c: Log is configured to *not* fsync() on all Append() calls
I20260812 17:49:08.904631 30756 raft_consensus.cc:348] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45567 } } peers { permanent_uuid: "2ac4321512824119b23120814640f1ec" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 44795 } } peers { permanent_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33651 } }
I20260812 17:49:08.904901 30756 raft_consensus.cc:374] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:49:08.904915 30758 tablet_bootstrap.cc:492] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c: No bootstrap required, opened a new log
I20260812 17:49:08.904917 30757 tablet_bootstrap.cc:492] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec: No bootstrap required, opened a new log
I20260812 17:49:08.904984 30756 raft_consensus.cc:729] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2e91d9dbe74e16ade0422e4fe387fa, State: Initialized, Role: FOLLOWER
I20260812 17:49:08.905128 30758 ts_tablet_manager.cc:1397] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:49:08.905128 30757 ts_tablet_manager.cc:1397] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260812 17:49:08.905547 30756 consensus_queue.cc:260] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45567 } } peers { permanent_uuid: "2ac4321512824119b23120814640f1ec" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 44795 } } peers { permanent_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33651 } }
I20260812 17:49:08.906773 30756 ts_tablet_manager.cc:1428] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 17:49:08.907177 30523 heartbeater.cc:503] Master 127.13.38.62:35887 was elected leader, sending a full tablet report...
I20260812 17:49:08.909401 30758 raft_consensus.cc:348] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45567 } } peers { permanent_uuid: "2ac4321512824119b23120814640f1ec" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 44795 } } peers { permanent_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33651 } }
I20260812 17:49:08.909744 30758 raft_consensus.cc:374] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:49:08.909813 30758 raft_consensus.cc:729] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b5b42ed8f414fd483555fb7fb775b1c, State: Initialized, Role: FOLLOWER
I20260812 17:49:08.909725 30757 raft_consensus.cc:348] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45567 } } peers { permanent_uuid: "2ac4321512824119b23120814640f1ec" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 44795 } } peers { permanent_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33651 } }
I20260812 17:49:08.909991 30757 raft_consensus.cc:374] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:49:08.910115 30757 raft_consensus.cc:729] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac4321512824119b23120814640f1ec, State: Initialized, Role: FOLLOWER
I20260812 17:49:08.910392 30758 consensus_queue.cc:260] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45567 } } peers { permanent_uuid: "2ac4321512824119b23120814640f1ec" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 44795 } } peers { permanent_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33651 } }
I20260812 17:49:08.910526 30757 consensus_queue.cc:260] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45567 } } peers { permanent_uuid: "2ac4321512824119b23120814640f1ec" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 44795 } } peers { permanent_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33651 } }
I20260812 17:49:08.911191 30730 heartbeater.cc:503] Master 127.13.38.62:35887 was elected leader, sending a full tablet report...
I20260812 17:49:08.911584 30757 ts_tablet_manager.cc:1428] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260812 17:49:08.911584 30758 ts_tablet_manager.cc:1428] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260812 17:49:08.911931 30279 heartbeater.cc:503] Master 127.13.38.62:35887 was elected leader, sending a full tablet report...
I20260812 17:49:08.999759 30764 raft_consensus.cc:482] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:49:09.000052 30764 raft_consensus.cc:504] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45567 } } peers { permanent_uuid: "2ac4321512824119b23120814640f1ec" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 44795 } } peers { permanent_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33651 } }
I20260812 17:49:09.001407 30764 leader_election.cc:288] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd2e91d9dbe74e16ade0422e4fe387fa (127.13.38.2:45567), 2ac4321512824119b23120814640f1ec (127.13.38.1:44795)
I20260812 17:49:09.006407 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:09.001699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:44795 (local address 127.13.38.3:32869)
0812 17:49:09.001816 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:09.001832 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:09.001940 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:09.002312 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:09.002315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:09.002356 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:49:09.002510 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:09.002516 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.003678 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.003680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:09.005052 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:09.005059 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.005853 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.005859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:09.005977 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:09.006052 (+ 75us) client_negotiation.cc:769] Sending connection context
0812 17:49:09.006146 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 17:49:09.006239 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":6}
I20260812 17:49:09.006394 30752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:09.001980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:44801 (local address 127.13.38.2:45567)
0812 17:49:09.002198 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:09.002205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:09.002331 (+ 126us) server_negotiation.cc:407] Connection header received
0812 17:49:09.002500 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:09.002502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:09.002544 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:09.002656 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:49:09.002910 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.003725 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.005220 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.005740 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.005905 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:09.005975 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:09.006174 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 17:49:09.006250 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260812 17:49:09.006402 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:09.001866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:32869 (local address 127.13.38.1:44795)
0812 17:49:09.002021 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:09.002027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:09.002045 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:49:09.002111 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:09.002114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:09.002170 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:09.002310 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:49:09.002637 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.003550 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.005237 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.005697 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.005789 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:09.005880 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:09.006193 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 17:49:09.006279 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 17:49:09.006682 30767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:09.001699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:45567 (local address 127.13.38.3:44801)
0812 17:49:09.002198 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:09.002209 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:09.002372 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:09.002681 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:09.002683 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:09.002699 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:49:09.002790 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:09.002794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.003822 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.003824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:09.005022 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:09.005036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.005943 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.005946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:09.006034 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:09.006054 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 17:49:09.006157 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 17:49:09.006239 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":79,"spinlock_wait_cycles":76032,"thread_start_us":165,"threads_started":1}
I20260812 17:49:09.006660 30763 raft_consensus.cc:482] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:49:09.006862 30763 raft_consensus.cc:504] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45567 } } peers { permanent_uuid: "2ac4321512824119b23120814640f1ec" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 44795 } } peers { permanent_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33651 } }
I20260812 17:49:09.007283 30203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73049317cf7244e7acaa237af33346a4" candidate_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ac4321512824119b23120814640f1ec" is_pre_election: true
I20260812 17:49:09.007340 30443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73049317cf7244e7acaa237af33346a4" candidate_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" is_pre_election: true
I20260812 17:49:09.007570 30443 raft_consensus.cc:2454] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b5b42ed8f414fd483555fb7fb775b1c in term 0.
I20260812 17:49:09.007591 30203 raft_consensus.cc:2454] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b5b42ed8f414fd483555fb7fb775b1c in term 0.
I20260812 17:49:09.008129 30554 leader_election.cc:302] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b5b42ed8f414fd483555fb7fb775b1c, cd2e91d9dbe74e16ade0422e4fe387fa; no voters:
I20260812 17:49:09.008291 30763 leader_election.cc:288] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ac4321512824119b23120814640f1ec (127.13.38.1:44795), 0b5b42ed8f414fd483555fb7fb775b1c (127.13.38.3:33651)
I20260812 17:49:09.008512 30764 raft_consensus.cc:2781] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:49:09.008611 30764 raft_consensus.cc:482] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:49:09.008663 30764 raft_consensus.cc:3037] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:49:09.010190 30764 raft_consensus.cc:504] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45567 } } peers { permanent_uuid: "2ac4321512824119b23120814640f1ec" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 44795 } } peers { permanent_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33651 } }
I20260812 17:49:09.010948 30203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73049317cf7244e7acaa237af33346a4" candidate_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ac4321512824119b23120814640f1ec"
I20260812 17:49:09.011107 30203 raft_consensus.cc:3037] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:49:09.011657 30443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73049317cf7244e7acaa237af33346a4" candidate_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa"
I20260812 17:49:09.011777 30443 raft_consensus.cc:3037] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:49:09.013154 30443 raft_consensus.cc:2454] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b5b42ed8f414fd483555fb7fb775b1c in term 1.
I20260812 17:49:09.013304 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:09.008635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:33651 (local address 127.13.38.2:52429)
0812 17:49:09.008908 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:09.008926 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:09.009015 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:09.009584 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:09.009587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:09.009605 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:49:09.009718 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:09.009728 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.010646 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.010650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:09.011898 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:09.011910 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.012893 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.012897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:09.013001 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:09.013023 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 17:49:09.013087 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 17:49:09.013163 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":29}
I20260812 17:49:09.013509 30554 leader_election.cc:302] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b5b42ed8f414fd483555fb7fb775b1c, cd2e91d9dbe74e16ade0422e4fe387fa; no voters:
I20260812 17:49:09.013765 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:09.009152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:52429 (local address 127.13.38.3:33651)
0812 17:49:09.009277 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:09.009281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:09.009293 (+ 12us) server_negotiation.cc:407] Connection header received
0812 17:49:09.009336 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:09.009338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:09.009383 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:09.009450 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:49:09.009836 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.010509 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.012053 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.012675 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.012769 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:09.012833 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:09.013569 (+ 736us) server_negotiation.cc:299] Negotiation successful
0812 17:49:09.013650 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 17:49:09.014040 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:09.009063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:49979 (local address 127.13.38.1:44795)
0812 17:49:09.009194 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:09.009198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:09.009409 (+ 211us) server_negotiation.cc:407] Connection header received
0812 17:49:09.009552 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:09.009557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:09.009607 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:09.009691 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:49:09.009998 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.010801 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.012552 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.013189 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.013275 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:09.013351 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:09.013755 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 17:49:09.013852 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 17:49:09.014381 30768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:09.008708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:44795 (local address 127.13.38.2:49979)
0812 17:49:09.009310 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:09.009321 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:09.009445 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:09.009705 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:09.009708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:09.009725 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:49:09.009851 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:09.009857 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.011186 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.011189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:09.012380 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:09.012391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.013475 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.013479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:09.013594 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:09.013620 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 17:49:09.014167 (+ 547us) client_negotiation.cc:240] Negotiation successful
0812 17:49:09.014244 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":5,"thread_start_us":151,"threads_started":1}
I20260812 17:49:09.014318 30660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73049317cf7244e7acaa237af33346a4" candidate_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" is_pre_election: true
I20260812 17:49:09.014586 30660 raft_consensus.cc:2379] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd2e91d9dbe74e16ade0422e4fe387fa in current term 1: Already voted for candidate 0b5b42ed8f414fd483555fb7fb775b1c in this term.
I20260812 17:49:09.014935 30201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73049317cf7244e7acaa237af33346a4" candidate_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ac4321512824119b23120814640f1ec" is_pre_election: true
I20260812 17:49:09.015157 30203 raft_consensus.cc:2454] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b5b42ed8f414fd483555fb7fb775b1c in term 1.
I20260812 17:49:09.015319 30764 leader_election.cc:288] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [CANDIDATE]: Term 1 election: Requested vote from peers cd2e91d9dbe74e16ade0422e4fe387fa (127.13.38.2:45567), 2ac4321512824119b23120814640f1ec (127.13.38.1:44795)
I20260812 17:49:09.015432 30764 raft_consensus.cc:2781] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:49:09.015548 30764 raft_consensus.cc:686] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 1 LEADER]: Becoming Leader. State: Replica: 0b5b42ed8f414fd483555fb7fb775b1c, State: Running, Role: LEADER
I20260812 17:49:09.015748 30333 leader_election.cc:302] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd2e91d9dbe74e16ade0422e4fe387fa; no voters: 0b5b42ed8f414fd483555fb7fb775b1c, 2ac4321512824119b23120814640f1ec
I20260812 17:49:09.015918 30764 consensus_queue.cc:237] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45567 } } peers { permanent_uuid: "2ac4321512824119b23120814640f1ec" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 44795 } } peers { permanent_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33651 } }
I20260812 17:49:09.016223 30763 raft_consensus.cc:2726] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 17:49:09.019395 29961 catalog_manager.cc:5495] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b5b42ed8f414fd483555fb7fb775b1c (127.13.38.3). New cstate: current_term: 1 leader_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ac4321512824119b23120814640f1ec" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 44795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b5b42ed8f414fd483555fb7fb775b1c" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33651 } health_report { overall_health: HEALTHY } } }
I20260812 17:49:09.038910 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:09.034704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57476 (local address 127.13.38.3:33651)
0812 17:49:09.034859 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:09.034863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:09.034880 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:49:09.035012 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:09.035014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:09.035082 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:09.035202 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:49:09.035505 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.036129 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.037062 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:09.037274 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:09.037405 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:09.037823 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:49:09.038385 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:49:09.038499 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:09.038635 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 17:49:09.038718 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 17:49:09.045941 30201 raft_consensus.cc:1261] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec [term 1 FOLLOWER]: Refusing update from remote peer 0b5b42ed8f414fd483555fb7fb775b1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:49:09.045992 30443 raft_consensus.cc:1261] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [term 1 FOLLOWER]: Refusing update from remote peer 0b5b42ed8f414fd483555fb7fb775b1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:49:09.046834 30771 consensus_queue.cc:1035] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ac4321512824119b23120814640f1ec" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 44795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:49:09.047072 30764 consensus_queue.cc:1035] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd2e91d9dbe74e16ade0422e4fe387fa" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:49:09.057341 30775 mvcc.cc:205] Tried to move back new op lower bound from 7317737263283400704 to 7317737263172558848. Current Snapshot: MvccSnapshot[applied={T|T < 7317737263283400704}]
I20260812 17:49:09.058903 30778 mvcc.cc:205] Tried to move back new op lower bound from 7317737263283400704 to 7317737263172558848. Current Snapshot: MvccSnapshot[applied={T|T < 7317737263283400704}]
I20260812 17:49:09.067883 29961 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:49:09.068207 29961 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:49:09.070811 29961 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d7dca8308eb4235a32e2338e4f12cec: Sending DeleteTablet for 3 replicas of tablet 73049317cf7244e7acaa237af33346a4
I20260812 17:49:09.071724 30411 tablet_service.cc:1514] Processing DeleteTablet for tablet 73049317cf7244e7acaa237af33346a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:49:09 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:52166
I20260812 17:49:09.071837 30170 tablet_service.cc:1514] Processing DeleteTablet for tablet 73049317cf7244e7acaa237af33346a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:49:09 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:35436
I20260812 17:49:09.072320 30791 tablet_replica.cc:330] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa: stopping tablet replica
I20260812 17:49:09.072431 30795 tablet_replica.cc:330] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec: stopping tablet replica
I20260812 17:49:09.072764 30791 raft_consensus.cc:2229] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:49:09.072768 30795 raft_consensus.cc:2229] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:49:09.073086 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 30073
I20260812 17:49:09.073175 30795 raft_consensus.cc:2258] T 73049317cf7244e7acaa237af33346a4 P 2ac4321512824119b23120814640f1ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:49:09.078487 30791 raft_consensus.cc:2258] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:49:09.079708 30624 tablet_service.cc:1514] Processing DeleteTablet for tablet 73049317cf7244e7acaa237af33346a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:49:09 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:57472
I20260812 17:49:09.080595 30796 tablet_replica.cc:330] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c: stopping tablet replica
I20260812 17:49:09.080900 30791 ts_tablet_manager.cc:1905] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:49:09.080967 30796 raft_consensus.cc:2229] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:49:09.081430 30796 raft_consensus.cc:2258] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:49:09.082866 30796 ts_tablet_manager.cc:1905] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:49:09.084671 30791 ts_tablet_manager.cc:1918] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:49:09.084854 30791 log.cc:1196] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/73049317cf7244e7acaa237af33346a4
I20260812 17:49:09.085665 30791 ts_tablet_manager.cc:1939] T 73049317cf7244e7acaa237af33346a4 P cd2e91d9dbe74e16ade0422e4fe387fa: Deleting consensus metadata
I20260812 17:49:09.085983 30796 ts_tablet_manager.cc:1918] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:49:09.086097 30796 log.cc:1196] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/73049317cf7244e7acaa237af33346a4
I20260812 17:49:09.086443 30796 ts_tablet_manager.cc:1939] T 73049317cf7244e7acaa237af33346a4 P 0b5b42ed8f414fd483555fb7fb775b1c: Deleting consensus metadata
W20260812 17:49:09.086799 29950 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:44795 (error 108)
I20260812 17:49:09.086961 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 30288
I20260812 17:49:09.090091 29949 catalog_manager.cc:4841] TS 0b5b42ed8f414fd483555fb7fb775b1c (127.13.38.3:33651): tablet 73049317cf7244e7acaa237af33346a4 (table test-table [id=783ea851b8274b8e822b70f69f565d0d]) successfully deleted
I20260812 17:49:09.096812 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:09.096429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:44795 (local address 127.0.0.1:35450)
0812 17:49:09.096605 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:09.096701 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:44795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260812 17:49:09.097090 29950 catalog_manager.cc:4584] TS 2ac4321512824119b23120814640f1ec (127.13.38.1:44795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73049317cf7244e7acaa237af33346a4: Network error: Client connection negotiation failed: client connection to 127.13.38.1:44795: connect: Connection refused (error 111)
I20260812 17:49:09.099236 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 30529
I20260812 17:49:09.105067 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:09.099333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:45567 (local address 127.0.0.1:52170)
0812 17:49:09.104877 (+ 5544us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:09.104950 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.2:45567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5426}
I20260812 17:49:09.116478 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 29912
2026-08-12T17:49:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2788 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:49:09 dist-test-slave-sb2z krb5kdc[30802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:49:09 dist-test-slave-sb2z krb5kdc[30802](info): set up 2 sockets
Aug 12 17:49:09 dist-test-slave-sb2z krb5kdc[30802](info): commencing operation
krb5kdc: starting...
W20260812 17:49:10.514685 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.363s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:49:10 dist-test-slave-sb2z krb5kdc[30802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:49:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:49:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.38.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.38.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/rangeradmin_127.13.38.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/rangeradmin_127.13.38.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.38.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.38.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/rangerlookup_127.13.38.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/rangerlookup_127.13.38.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.38.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.38.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/HTTP_127.13.38.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/HTTP_127.13.38.20@KRBTEST.COM.keytab.
I20260812 17:49:10.822643 13464 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskuCZvAR/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/postgres -l logfile start
2026-08-12 17:49:13.668 UTC [30852] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 17:49:13.669 UTC [30852] LOG: listening on IPv4 address "127.13.38.20", port 57587
2026-08-12 17:49:13.673 UTC [30852] LOG: listening on Unix socket "/tmp/.s.PGSQL.57587"
2026-08-12 17:49:13.715 UTC [30855] LOG: database system was shut down at 2026-08-12 17:49:11 UTC
2026-08-12 17:49:13.719 UTC [30852] LOG: database system is ready to accept connections
I20260812 17:49:14.935741 13464 mini_postgres.cc:96] Postgres bound to 57587
2026-08-12 17:49:14.945 UTC [30870] FATAL: database "slave" does not exist
127.13.38.20:57587 - accepting connections
I20260812 17:49:14.946641 13464 mini_ranger.cc:161] Starting Ranger...
I20260812 17:49:14.965255 13464 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 17:49:15.519340 13464 mini_ranger.cc:87] Created ranger Postgres database
I20260812 17:49:15.519443 13464 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-admin
2026-08-12 17:49:15,914 [I] DB FLAVOR :POSTGRES
2026-08-12 17:49:15,914 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 17:49:15,914 [I] Checking connection..
2026-08-12 17:49:15,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 17:49:16,215 [I] Checking connection passed.
2026-08-12 17:49:16,216 [I] --------- Verifying version history table ---------
2026-08-12 17:49:16,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 17:49:16,481 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 17:49:16,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 17:49:16,771 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 17:49:16,772 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 17:49:16,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 17:49:17.044 UTC [30963] WARNING: there is no transaction in progress
2026-08-12 17:49:17,054 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 17:49:17,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 17:49:17,357 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 17:49:17,358 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 17:49:17,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 17:49:17,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 17:49:17,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sb2z.c.gcp-upstream.internal','N') ;"
2026-08-12 17:49:18,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 17:49:18,370 [I] Table x_portal_user does not exist in database ranger
2026-08-12 17:49:18,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 17:49:18,649 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 17:49:18,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 17:49:18,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 17:49:19,163 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 17:49:19,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 17:49:20.261 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.279 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.297 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.314 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.335 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.358 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.422 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.430 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.445 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.457 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.469 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.485 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.495 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.502 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.510 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:20.519 UTC [31137] WARNING: there is no transaction in progress
2026-08-12 17:49:21,271 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 17:49:21,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 17:49:21,533 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 17:49:21,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 17:49:21,811 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 17:49:21,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 17:49:22,091 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 17:49:22,092 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 17:49:22,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 17:49:22,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sb2z.c.gcp-upstream.internal';"
2026-08-12 17:49:22,855 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 17:49:22,856 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 17:49:23,134 [I] DB_PATCHES have already been applied
I20260812 17:49:23.141044 13464 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-admin:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskuCZvAR/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 17:49:23.141129 13464 mini_ranger.cc:193] Using host: 127.13.38.20
I20260812 17:49:23.146018 13464 mini_ranger.cc:227] Ranger admin URL: http://127.13.38.20:43667
Aug 12, 2026 5:49:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 5:49:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 5:49:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 5:49:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 5:49:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 5:49:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 5:49:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 17:49:24.565773 13464 mini_ranger.cc:160] Time spent starting Ranger: real 9.619s user 0.000s sys 0.014s
Aug 12, 2026 5:49:46 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 5:49:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 5:49:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 5:49:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 5:49:46 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 5:49:47 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 17:49:48.463320 13464 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.38.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.38.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/rangerkms_127.13.38.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/rangerkms_127.13.38.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.38.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.38.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.38.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/HTTP_127.13.38.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.38.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/HTTP_127.13.38.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 17:49:48 dist-test-slave-sb2z krb5kdc[30802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556988, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 17:49:48.581107 13464 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 17:49:48.593142 13464 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 17:49:49.660789 13464 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 17:49:49.660918 13464 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-kms
I20260812 17:49:49.661036 13464 mini_ranger_kms.cc:227] Using postgres at 127.13.38.20:57587
2026-08-12 17:49:50,126 [I] DB FLAVOR :POSTGRES
2026-08-12 17:49:50,126 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 17:49:50,126 [I] Checking connection
2026-08-12 17:49:50,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 17:49:50,390 [I] connection success
2026-08-12 17:49:50,390 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 17:49:50,390 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 17:49:50,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 17:49:50,662 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 17:49:50,662 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 17:49:50,662 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 17:49:50,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:57587/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 17:49:50,962 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 17:49:51.522747 13464 mini_ranger_kms.cc:310] Created kms service
I20260812 17:49:51.747119 13464 mini_ranger_kms.cc:326] Created kudu user
I20260812 17:49:51.803681 13464 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 17:49:51 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556988, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:49:52.219775 13464 mini_ranger_kms.cc:384] Added ranger policy
I20260812 17:49:52.219949 13464 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-kms:/tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 17:49:52.219975 13464 mini_ranger_kms.cc:240] Using host: 127.13.38.20
I20260812 17:49:52.225906 13464 mini_ranger_kms.cc:276] Ranger KMS PID: 31425
I20260812 17:49:52.226063 13464 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.38.20:38045
Aug 12, 2026 5:49:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 5:49:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 5:49:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 5:49:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 5:49:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 5:49:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 17:49:54 dist-test-slave-sb2z krb5kdc[30802](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786556994, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.38.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:49:55.128798 13464 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.548s user 0.025s sys 0.347s
I20260812 17:49:55.128957 13464 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 17:49:55.129096 13464 mini_ranger_kms.cc:401] 127.13.38.20:38045/kms/v1/keys
Aug 12 17:49:55 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556994, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.38.20@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
Aug 12 17:49:55 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556994, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.38.20@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
Aug 12 17:49:58 dist-test-slave-sb2z krb5kdc[30802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:49:58.184518 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:44589
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:44589
--ranger_config_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.38.20:38045/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:49:58.316093 31517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:58.316375 31517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:58.316417 31517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:58.320688 31517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:49:58.320760 31517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:58.320780 31517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:58.320797 31517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:49:58.320812 31517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:49:58.326808 31517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.38.20:38045/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:44589
--ranger_config_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:44589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:58.328152 31517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:58.329308 31517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:58.339557 31529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:58.339641 31517 server_base.cc:1034] running on GCE node
W20260812 17:49:58.339644 31531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:58.339882 31535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:58.340613 31517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:58.341877 31517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:58.343097 31517 hybrid_clock.cc:648] HybridClock initialized: now 1786556998343084 us; error 38 us; skew 500 ppm
Aug 12 17:49:58 dist-test-slave-sb2z krb5kdc[30802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556998, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:49:58.346616 31517 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:49:58.347770 31517 webserver.cc:460] Webserver started at http://127.13.38.62:34705/ using document root <none> and password file <none>
I20260812 17:49:58.348392 31517 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:58.348484 31517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:58.348721 31517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 17:49:58 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556998, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:49:58.398279 31517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "273932c455b844428e43b30713dd4526"
format_stamp: "Formatted at 2026-08-12 17:49:58 on dist-test-slave-sb2z"
server_key: "860d689a08a1ecc3102326fba9b51624"
server_key_iv: "bed8a6cabb53c7569ca51d354d9ef538"
server_key_version: "kuduclusterkey@0"
I20260812 17:49:58.398964 31517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "273932c455b844428e43b30713dd4526"
format_stamp: "Formatted at 2026-08-12 17:49:58 on dist-test-slave-sb2z"
server_key: "860d689a08a1ecc3102326fba9b51624"
server_key_iv: "bed8a6cabb53c7569ca51d354d9ef538"
server_key_version: "kuduclusterkey@0"
I20260812 17:49:58.403328 31517 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 17:49:58.417708 31547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:58.419260 31517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:49:58.419445 31517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "273932c455b844428e43b30713dd4526"
format_stamp: "Formatted at 2026-08-12 17:49:58 on dist-test-slave-sb2z"
server_key: "860d689a08a1ecc3102326fba9b51624"
server_key_iv: "bed8a6cabb53c7569ca51d354d9ef538"
server_key_version: "kuduclusterkey@0"
I20260812 17:49:58.419636 31517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:58.452453 31517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:58.455860 31517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:58.456149 31517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:58.465724 31517 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:44589
I20260812 17:49:58.466290 31637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:44589 every 8 connection(s)
I20260812 17:49:58.466993 31517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:49:58.470129 31639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:49:58.474893 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 31517
I20260812 17:49:58.475029 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:49:58.475322 13464 external_mini_cluster.cc:1428] Setting key ac2742b0228bc6e93a090cd1839f3c0e
I20260812 17:49:58.476306 31639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526: Bootstrap starting.
I20260812 17:49:58.479394 31639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526: Neither blocks nor log segments found. Creating new log.
I20260812 17:49:58.480388 31639 log.cc:824] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526: Log is configured to *not* fsync() on all Append() calls
I20260812 17:49:58.482759 31639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526: No bootstrap required, opened a new log
Aug 12 17:49:58 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:49:58.487787 31639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273932c455b844428e43b30713dd4526" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44589 } }
I20260812 17:49:58.488145 31639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:49:58.488268 31639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273932c455b844428e43b30713dd4526, State: Initialized, Role: FOLLOWER
I20260812 17:49:58.488891 31639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273932c455b844428e43b30713dd4526" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44589 } }
I20260812 17:49:58.489073 31639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:49:58.489163 31639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:49:58.489305 31639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:49:58.490658 31639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273932c455b844428e43b30713dd4526" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44589 } }
I20260812 17:49:58.491058 31639 leader_election.cc:302] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 273932c455b844428e43b30713dd4526; no voters:
I20260812 17:49:58.491451 31639 leader_election.cc:288] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:49:58.491590 31644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:49:58.492012 31644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [term 1 LEADER]: Becoming Leader. State: Replica: 273932c455b844428e43b30713dd4526, State: Running, Role: LEADER
I20260812 17:49:58.492406 31644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273932c455b844428e43b30713dd4526" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44589 } }
I20260812 17:49:58.493034 31639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:49:58.493518 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:58.476947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50886 (local address 127.13.38.62:44589)
0812 17:49:58.477425 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:58.477445 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:58.477487 (+ 42us) server_negotiation.cc:407] Connection header received
0812 17:49:58.478373 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:58.478397 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:58.478823 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:58.479236 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:58.480227 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:58.481585 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:58.482932 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:58.483675 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:58.483841 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:58.487195 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:58.487202 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:58.487223 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:49:58.487261 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:58.489443 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:58.490009 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:58.490016 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:58.490023 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:58.490171 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:58.490526 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:58.490529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:58.490531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:58.491095 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:58.492015 (+ 920us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:58.492177 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 17:49:58.492496 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":136,"threads_started":1}
I20260812 17:49:58.493777 31644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "273932c455b844428e43b30713dd4526" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273932c455b844428e43b30713dd4526" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44589 } } }
I20260812 17:49:58.493971 31644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [sys.catalog]: This master's current role is: LEADER
I20260812 17:49:58.494259 31644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 273932c455b844428e43b30713dd4526. Latest consensus state: current_term: 1 leader_uuid: "273932c455b844428e43b30713dd4526" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "273932c455b844428e43b30713dd4526" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44589 } } }
I20260812 17:49:58.494423 31644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526 [sys.catalog]: This master's current role is: LEADER
I20260812 17:49:58.494769 31658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:49:58.498692 31658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:49:58.499624 31639 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 17:49:58.509681 31658 catalog_manager.cc:1347] Generated new cluster ID: 56a34c987c6544a0b6cf8ecbc0f5d214
I20260812 17:49:58.509845 31658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:49:58.521831 31658 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:49:58.523327 31658 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:49:58.529867 31658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526: Generated new TSK 0
I20260812 17:49:58.530897 31658 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:49:59.347599 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.38.20:38045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44589
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:49:59.490614 31713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:59.491024 31713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:59.491122 31713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:59.496160 31713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:59.496243 31713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:59.496306 31713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:49:59.502360 31713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.38.20:38045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:59.503679 31713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:59.504850 31713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:59.519670 31723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:59.519721 31729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:59.519840 31727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:59.521296 31713 server_base.cc:1034] running on GCE node
I20260812 17:49:59.521888 31713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:59.522797 31713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:59.524053 31713 hybrid_clock.cc:648] HybridClock initialized: now 1786556999523999 us; error 87 us; skew 500 ppm
Aug 12 17:49:59 dist-test-slave-sb2z krb5kdc[30802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556999, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:49:59.528211 31713 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:49:59.529318 31713 webserver.cc:460] Webserver started at http://127.13.38.1:36669/ using document root <none> and password file <none>
I20260812 17:49:59.530045 31713 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:59.530161 31713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:59.530447 31713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 17:49:59 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556999, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:49:59.548233 31713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "02848c8f2aca41a2960fab89cfd71a6a"
format_stamp: "Formatted at 2026-08-12 17:49:59 on dist-test-slave-sb2z"
server_key: "0396f11b88c32b715e3710d4dea2f99b"
server_key_iv: "15a49a46ac8fcfc5f236e6c3b7ea62d4"
server_key_version: "kuduclusterkey@0"
I20260812 17:49:59.549005 31713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "02848c8f2aca41a2960fab89cfd71a6a"
format_stamp: "Formatted at 2026-08-12 17:49:59 on dist-test-slave-sb2z"
server_key: "0396f11b88c32b715e3710d4dea2f99b"
server_key_iv: "15a49a46ac8fcfc5f236e6c3b7ea62d4"
server_key_version: "kuduclusterkey@0"
I20260812 17:49:59.554808 31713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 17:49:59.572917 31745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:59.576961 31713 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260812 17:49:59.577255 31713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "02848c8f2aca41a2960fab89cfd71a6a"
format_stamp: "Formatted at 2026-08-12 17:49:59 on dist-test-slave-sb2z"
server_key: "0396f11b88c32b715e3710d4dea2f99b"
server_key_iv: "15a49a46ac8fcfc5f236e6c3b7ea62d4"
server_key_version: "kuduclusterkey@0"
I20260812 17:49:59.577548 31713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:59.594779 31713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:59.598433 31713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:59.598673 31713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:59.599385 31713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:49:59.600432 31713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:49:59.600512 31713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:59.600589 31713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:49:59.600636 31713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:59.612216 31713 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:41489
I20260812 17:49:59.612290 31920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:41489 every 8 connection(s)
I20260812 17:49:59.613584 31713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:49:59.622041 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 31713
I20260812 17:49:59.622238 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:49:59.622603 13464 external_mini_cluster.cc:1428] Setting key 29bcdb31a2e9015b741d3afef488d3b1
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
Aug 12 17:49:59 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556999, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:49:59.647588 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:59.615869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:43313 (local address 127.13.38.62:44589)
0812 17:49:59.617672 (+ 1803us) server_negotiation.cc:206] Beginning negotiation
0812 17:49:59.617677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:49:59.619450 (+ 1773us) server_negotiation.cc:407] Connection header received
0812 17:49:59.619509 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:49:59.619512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:49:59.619597 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:49:59.619688 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:49:59.622543 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:59.623335 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:59.625054 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:49:59.625471 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:49:59.625573 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:59.642257 (+ 16684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:49:59.642265 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:49:59.642271 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:49:59.642313 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:49:59.644469 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:59.645261 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:59.645265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:59.645267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:59.645378 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:49:59.645909 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:49:59.645914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:49:59.645915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:49:59.646190 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:49:59.646289 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 17:49:59.647205 (+ 916us) server_negotiation.cc:299] Negotiation successful
0812 17:49:59.647362 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1705,"thread_start_us":106,"threads_started":1}
I20260812 17:49:59.648676 31930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:49:59.616424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44589 (local address 127.13.38.1:43313)
0812 17:49:59.617968 (+ 1544us) negotiation.cc:107] Waiting for socket to connect
0812 17:49:59.618030 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 17:49:59.619120 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:49:59.620027 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:49:59.620036 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:49:59.620496 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:49:59.621279 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:59.621302 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:59.623517 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:59.623520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:59.624896 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:49:59.624912 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:49:59.625830 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:49:59.625837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:49:59.626159 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:49:59.639454 (+ 13295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:49:59.639486 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:49:59.641937 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:49:59.644680 (+ 2743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:49:59.644688 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:49:59.644712 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:49:59.645119 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:49:59.645527 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:49:59.645530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:49:59.645532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:49:59.645776 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:49:59.646397 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:49:59.646405 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:49:59.646843 (+ 438us) client_negotiation.cc:769] Sending connection context
0812 17:49:59.647105 (+ 262us) client_negotiation.cc:240] Negotiation successful
0812 17:49:59.647498 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1301,"thread_start_us":159,"threads_started":1}
I20260812 17:49:59.650149 31921 heartbeater.cc:347] Connected to a master server at 127.13.38.62:44589
I20260812 17:49:59.650468 31921 heartbeater.cc:464] Registering TS with master...
I20260812 17:49:59.651142 31921 heartbeater.cc:511] Master 127.13.38.62:44589 requested a full tablet report, sending...
I20260812 17:49:59.653194 31568 ts_manager.cc:194] Registered new tserver with Master: 02848c8f2aca41a2960fab89cfd71a6a (127.13.38.1:41489)
I20260812 17:49:59.654621 31568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:43313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:49:59.672850 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.38.20:38045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44589
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 17:49:59 dist-test-slave-sb2z krb5kdc[30802](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786556999, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 17:49:59.841831 31940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:49:59.842283 31940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:49:59.842425 31940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:49:59.848726 31940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:49:59.848824 31940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:49:59.848922 31940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:49:59.857492 31940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.38.20:38045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:49:59.859388 31940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:49:59.860977 31940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:49:59.873998 31949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:59.874008 31950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:49:59.874078 31952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:49:59.874430 31940 server_base.cc:1034] running on GCE node
I20260812 17:49:59.874847 31940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:49:59.875602 31940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:49:59.876794 31940 hybrid_clock.cc:648] HybridClock initialized: now 1786556999876757 us; error 51 us; skew 500 ppm
Aug 12 17:49:59 dist-test-slave-sb2z krb5kdc[30802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556999, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:49:59.880254 31940 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:49:59.881436 31940 webserver.cc:460] Webserver started at http://127.13.38.2:41121/ using document root <none> and password file <none>
I20260812 17:49:59.882191 31940 fs_manager.cc:359] Metadata directory not provided
I20260812 17:49:59.882289 31940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:49:59.882604 31940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 17:49:59 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556999, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:49:59.898721 31940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "c4147f3f3e4643a2972bece4e7c93de3"
format_stamp: "Formatted at 2026-08-12 17:49:59 on dist-test-slave-sb2z"
server_key: "b85ecc7c44ebca8ed67bb88d0a88c167"
server_key_iv: "34bbf4341d7e1f7135b799bc13916626"
server_key_version: "kuduclusterkey@0"
I20260812 17:49:59.899519 31940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "c4147f3f3e4643a2972bece4e7c93de3"
format_stamp: "Formatted at 2026-08-12 17:49:59 on dist-test-slave-sb2z"
server_key: "b85ecc7c44ebca8ed67bb88d0a88c167"
server_key_iv: "34bbf4341d7e1f7135b799bc13916626"
server_key_version: "kuduclusterkey@0"
I20260812 17:49:59.904695 31940 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:49:59.922375 31961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:59.924372 31940 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 17:49:59.924604 31940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "c4147f3f3e4643a2972bece4e7c93de3"
format_stamp: "Formatted at 2026-08-12 17:49:59 on dist-test-slave-sb2z"
server_key: "b85ecc7c44ebca8ed67bb88d0a88c167"
server_key_iv: "34bbf4341d7e1f7135b799bc13916626"
server_key_version: "kuduclusterkey@0"
I20260812 17:49:59.924774 31940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:49:59.958410 31940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:49:59.963222 31940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:49:59.963616 31940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:49:59.964545 31940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:49:59.965811 31940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:49:59.965943 31940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:59.966068 31940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:49:59.966167 31940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:49:59.992689 31940 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:45129
I20260812 17:49:59.994527 31940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:49:59.995719 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 31940
I20260812 17:49:59.995857 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:49:59.995721 32150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:45129 every 8 connection(s)
I20260812 17:49:59.996181 13464 external_mini_cluster.cc:1428] Setting key 9274e6566ec1e0a4fc5192a720a2eb4d
Aug 12 17:50:00 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556999, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:50:00.020272 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.002912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:59489 (local address 127.13.38.62:44589)
0812 17:50:00.003070 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:00.003075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:00.003709 (+ 634us) server_negotiation.cc:407] Connection header received
0812 17:50:00.005447 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:00.005451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:00.005515 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:00.005577 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:00.007644 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.008534 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.010394 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.010766 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.010855 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.014596 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:00.014600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:00.014603 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:00.014637 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:00.016733 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:00.017656 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:00.017663 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:00.017666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:00.017801 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:00.018473 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:00.018478 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:00.018480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:00.018749 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:00.018866 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:00.019824 (+ 958us) server_negotiation.cc:299] Negotiation successful
0812 17:50:00.020030 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:50:00.022630 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.001617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44589 (local address 127.13.38.2:59489)
0812 17:50:00.003493 (+ 1876us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:00.003587 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:00.005142 (+ 1555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:00.006043 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:00.006067 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:00.006601 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:00.007411 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.007436 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.008716 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.008719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.010235 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.010248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.011110 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.011115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.011375 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.012204 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:00.012228 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:00.014358 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:00.016920 (+ 2562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:00.016929 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:00.016950 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:00.017476 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:00.018006 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:00.018009 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:00.018011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:00.018252 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:00.018957 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:00.018966 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:00.019455 (+ 489us) client_negotiation.cc:769] Sending connection context
0812 17:50:00.020995 (+ 1540us) client_negotiation.cc:240] Negotiation successful
0812 17:50:00.021472 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1585,"thread_start_us":252,"threads_started":1}
I20260812 17:50:00.024565 32151 heartbeater.cc:347] Connected to a master server at 127.13.38.62:44589
I20260812 17:50:00.024852 32151 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:00.025490 32151 heartbeater.cc:511] Master 127.13.38.62:44589 requested a full tablet report, sending...
I20260812 17:50:00.026814 31568 ts_manager.cc:194] Registered new tserver with Master: c4147f3f3e4643a2972bece4e7c93de3 (127.13.38.2:45129)
I20260812 17:50:00.027768 31568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:59489
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:50:00.064576 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.38.20:38045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44589
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:50:00.220503 32163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:50:00.220846 32163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:50:00.220968 32163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:50:00.226887 32163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:50:00.227005 32163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:50:00.227111 32163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:50:00.234163 32163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.38.20:38045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:50:00.235771 32163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:50:00.236928 32163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:50:00.248020 32170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:00.248543 32168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:00.248817 32163 server_base.cc:1034] running on GCE node
W20260812 17:50:00.250259 32172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:00.250804 32163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:50:00.251600 32163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:50:00.252785 32163 hybrid_clock.cc:648] HybridClock initialized: now 1786557000252745 us; error 59 us; skew 500 ppm
Aug 12 17:50:00 dist-test-slave-sb2z krb5kdc[30802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557000, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:00.255786 32163 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:50:00.256836 32163 webserver.cc:460] Webserver started at http://127.13.38.3:46503/ using document root <none> and password file <none>
I20260812 17:50:00.257443 32163 fs_manager.cc:359] Metadata directory not provided
I20260812 17:50:00.257541 32163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:50:00.257778 32163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 17:50:00 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557000, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:00.281639 32163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "1039dbc721ac47d0a5d7751276962664"
format_stamp: "Formatted at 2026-08-12 17:50:00 on dist-test-slave-sb2z"
server_key: "45d4684897eed4e6676106dc843d1d5c"
server_key_iv: "490885f271fcea0953032363fff70397"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:00.282688 32163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "1039dbc721ac47d0a5d7751276962664"
format_stamp: "Formatted at 2026-08-12 17:50:00 on dist-test-slave-sb2z"
server_key: "45d4684897eed4e6676106dc843d1d5c"
server_key_iv: "490885f271fcea0953032363fff70397"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:00.302071 32163 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.006s sys 0.001s
Aug 12 17:50:00 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556999, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:00.330672 32180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:00.332784 32163 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.001s
I20260812 17:50:00.333017 32163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "1039dbc721ac47d0a5d7751276962664"
format_stamp: "Formatted at 2026-08-12 17:50:00 on dist-test-slave-sb2z"
server_key: "45d4684897eed4e6676106dc843d1d5c"
server_key_iv: "490885f271fcea0953032363fff70397"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:00.333211 32163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:50:00.354076 32163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:50:00.358129 32163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:50:00.358408 32163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:50:00.359402 32163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:50:00.367995 32163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:50:00.368094 32163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:00.368165 32163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:50:00.368189 32163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:00.389266 32163 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:33201
I20260812 17:50:00.390658 32163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:50:00.392545 32361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:33201 every 8 connection(s)
I20260812 17:50:00.393290 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 32163
I20260812 17:50:00.393419 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:50:00.393736 13464 external_mini_cluster.cc:1428] Setting key 6ffe4262bdc4fecc4d4b2cf6ae173776
Aug 12 17:50:00 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557000, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:50:00.417668 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.397236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:34129 (local address 127.13.38.62:44589)
0812 17:50:00.397388 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:00.397393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:00.398667 (+ 1274us) server_negotiation.cc:407] Connection header received
0812 17:50:00.400094 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:00.400098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:00.400157 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:00.400255 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:00.402161 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.403089 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.404930 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.405380 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.405499 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.412595 (+ 7096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:00.412600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:00.412604 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:00.412636 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:00.414561 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:00.415311 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:00.415317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:00.415320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:00.415427 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:00.415984 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:00.415988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:00.415990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:00.416294 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:00.416405 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:00.417230 (+ 825us) server_negotiation.cc:299] Negotiation successful
0812 17:50:00.417461 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 17:50:00.418618 32369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.397823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44589 (local address 127.13.38.3:34129)
0812 17:50:00.398493 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:00.398562 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:00.399813 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:00.400536 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:00.400547 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:00.401187 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:00.401951 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.401974 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.403260 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.403264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.404782 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.404795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.405580 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.405587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.405801 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.410373 (+ 4572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:00.410397 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:00.412365 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:00.414712 (+ 2347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:00.414719 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:00.414742 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:00.415170 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:00.415611 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:00.415614 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:00.415617 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:00.415840 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:00.416426 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:00.416433 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:00.416839 (+ 406us) client_negotiation.cc:769] Sending connection context
0812 17:50:00.417086 (+ 247us) client_negotiation.cc:240] Negotiation successful
0812 17:50:00.417474 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":183,"threads_started":1}
I20260812 17:50:00.420351 32366 heartbeater.cc:347] Connected to a master server at 127.13.38.62:44589
I20260812 17:50:00.420611 32366 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:00.421262 32366 heartbeater.cc:511] Master 127.13.38.62:44589 requested a full tablet report, sending...
I20260812 17:50:00.422454 31567 ts_manager.cc:194] Registered new tserver with Master: 1039dbc721ac47d0a5d7751276962664 (127.13.38.3:33201)
I20260812 17:50:00.423388 31567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:34129
I20260812 17:50:00.432888 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:50:00.450232 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.435490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50898 (local address 127.13.38.62:44589)
0812 17:50:00.435649 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:00.435654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:00.439961 (+ 4307us) server_negotiation.cc:407] Connection header received
0812 17:50:00.440099 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:00.440103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:00.440165 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:00.440268 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:00.441462 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.442346 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.443729 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.444134 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.444217 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.446247 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:00.446251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:00.446254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:00.446283 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:00.448147 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:00.448763 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:00.448767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:00.448769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:00.448868 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:00.449221 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:00.449225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:00.449227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:00.449474 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:00.449613 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:00.449906 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 17:50:00.450062 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 17:50:00.454939 31567 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:50:00.461402 31567 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:50:00.494591 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.486272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:33201 (local address 127.0.0.1:38374)
0812 17:50:00.486726 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:00.486744 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:00.486846 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:00.488760 (+ 1914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:00.488768 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:00.488793 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:50:00.489073 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.489085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.490894 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.490897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.492432 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.492442 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.493847 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.493850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.494050 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.494136 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 17:50:00.494243 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 17:50:00.494396 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":288,"threads_started":1}
I20260812 17:50:00.495888 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.486948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.13.38.3:33201)
0812 17:50:00.487434 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:00.487487 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:00.487515 (+ 28us) server_negotiation.cc:407] Connection header received
0812 17:50:00.487636 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:00.487642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:00.488256 (+ 614us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:00.489268 (+ 1012us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:50:00.489429 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.490743 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.492675 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.493690 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.495275 (+ 1585us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.495499 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:00.495599 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 17:50:00.495711 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":265,"threads_started":1}
I20260812 17:50:00.496881 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.484270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:45129 (local address 127.0.0.1:36618)
0812 17:50:00.484826 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:00.484882 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:00.485094 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:00.489647 (+ 4553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:00.489651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:00.489671 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:50:00.489769 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.489775 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.492387 (+ 2612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.492391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.493843 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.493852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.496457 (+ 2605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.496461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.496564 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.496593 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 17:50:00.496649 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 17:50:00.496723 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":264,"threads_started":1}
I20260812 17:50:00.497800 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.485264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36618 (local address 127.13.38.2:45129)
0812 17:50:00.489101 (+ 3837us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:00.489122 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:00.489151 (+ 29us) server_negotiation.cc:407] Connection header received
0812 17:50:00.489238 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:00.489244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:00.489461 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:00.489943 (+ 482us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:50:00.490092 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.492244 (+ 2152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.494156 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.496290 (+ 2134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.497236 (+ 946us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.497428 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:00.497490 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 17:50:00.497600 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3377,"thread_start_us":205,"threads_started":1}
I20260812 17:50:00.499945 32269 tablet_service.cc:1467] Processing CreateTablet for tablet c2e26c29b70f4af1ba96b25547874ec1 (DEFAULT_TABLE table=test-table [id=035f4b137ee64f0b9592724ad02c8d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:50:00.501245 32269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e26c29b70f4af1ba96b25547874ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:50:00.501601 32046 tablet_service.cc:1467] Processing CreateTablet for tablet c2e26c29b70f4af1ba96b25547874ec1 (DEFAULT_TABLE table=test-table [id=035f4b137ee64f0b9592724ad02c8d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:50:00.503322 32046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e26c29b70f4af1ba96b25547874ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:50:00.504016 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.488000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:41489 (local address 127.0.0.1:46152)
0812 17:50:00.495016 (+ 7016us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:00.495031 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:00.495115 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:00.497218 (+ 2103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:00.497222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:00.497246 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:50:00.497377 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.497388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.501287 (+ 3899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.501292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.502394 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.502408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.503580 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.503584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.503693 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.503732 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 17:50:00.503787 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 17:50:00.503867 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6954,"thread_start_us":252,"threads_started":1}
I20260812 17:50:00.506034 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.488177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46152 (local address 127.13.38.1:41489)
0812 17:50:00.495271 (+ 7094us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:00.495279 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:00.495311 (+ 32us) server_negotiation.cc:407] Connection header received
0812 17:50:00.495402 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:00.495408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:00.496316 (+ 908us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:00.497574 (+ 1258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:50:00.497766 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.501127 (+ 3361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.502575 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.503395 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.505481 (+ 2086us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.505685 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:00.505770 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 17:50:00.505877 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6956,"thread_start_us":110,"threads_started":1}
I20260812 17:50:00.509559 31815 tablet_service.cc:1467] Processing CreateTablet for tablet c2e26c29b70f4af1ba96b25547874ec1 (DEFAULT_TABLE table=test-table [id=035f4b137ee64f0b9592724ad02c8d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:50:00.510790 31815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e26c29b70f4af1ba96b25547874ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:50:00.519238 32392 tablet_bootstrap.cc:492] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a: Bootstrap starting.
I20260812 17:50:00.521790 32392 tablet_bootstrap.cc:654] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a: Neither blocks nor log segments found. Creating new log.
I20260812 17:50:00.521907 32390 tablet_bootstrap.cc:492] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664: Bootstrap starting.
I20260812 17:50:00.522768 32392 log.cc:824] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a: Log is configured to *not* fsync() on all Append() calls
I20260812 17:50:00.523200 32391 tablet_bootstrap.cc:492] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3: Bootstrap starting.
I20260812 17:50:00.525411 32392 tablet_bootstrap.cc:492] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a: No bootstrap required, opened a new log
I20260812 17:50:00.525663 32392 ts_tablet_manager.cc:1397] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 17:50:00.528671 32390 tablet_bootstrap.cc:654] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664: Neither blocks nor log segments found. Creating new log.
I20260812 17:50:00.529733 32390 log.cc:824] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664: Log is configured to *not* fsync() on all Append() calls
I20260812 17:50:00.530845 32391 tablet_bootstrap.cc:654] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3: Neither blocks nor log segments found. Creating new log.
I20260812 17:50:00.531646 32392 raft_consensus.cc:348] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4147f3f3e4643a2972bece4e7c93de3" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45129 } } peers { permanent_uuid: "1039dbc721ac47d0a5d7751276962664" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33201 } } peers { permanent_uuid: "02848c8f2aca41a2960fab89cfd71a6a" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41489 } }
I20260812 17:50:00.532033 32392 raft_consensus.cc:374] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:50:00.532130 32392 raft_consensus.cc:729] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02848c8f2aca41a2960fab89cfd71a6a, State: Initialized, Role: FOLLOWER
I20260812 17:50:00.532599 32391 log.cc:824] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3: Log is configured to *not* fsync() on all Append() calls
I20260812 17:50:00.532711 32392 consensus_queue.cc:260] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4147f3f3e4643a2972bece4e7c93de3" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45129 } } peers { permanent_uuid: "1039dbc721ac47d0a5d7751276962664" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33201 } } peers { permanent_uuid: "02848c8f2aca41a2960fab89cfd71a6a" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41489 } }
I20260812 17:50:00.534026 32392 ts_tablet_manager.cc:1428] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a: Time spent starting tablet: real 0.008s user 0.002s sys 0.007s
I20260812 17:50:00.534433 31921 heartbeater.cc:503] Master 127.13.38.62:44589 was elected leader, sending a full tablet report...
I20260812 17:50:00.535245 32390 tablet_bootstrap.cc:492] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664: No bootstrap required, opened a new log
I20260812 17:50:00.535476 32390 ts_tablet_manager.cc:1397] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260812 17:50:00.536571 32391 tablet_bootstrap.cc:492] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3: No bootstrap required, opened a new log
I20260812 17:50:00.536772 32391 ts_tablet_manager.cc:1397] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260812 17:50:00.542339 32391 raft_consensus.cc:348] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4147f3f3e4643a2972bece4e7c93de3" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45129 } } peers { permanent_uuid: "1039dbc721ac47d0a5d7751276962664" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33201 } } peers { permanent_uuid: "02848c8f2aca41a2960fab89cfd71a6a" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41489 } }
I20260812 17:50:00.542718 32391 raft_consensus.cc:374] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:50:00.542789 32391 raft_consensus.cc:729] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4147f3f3e4643a2972bece4e7c93de3, State: Initialized, Role: FOLLOWER
I20260812 17:50:00.542861 32390 raft_consensus.cc:348] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4147f3f3e4643a2972bece4e7c93de3" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45129 } } peers { permanent_uuid: "1039dbc721ac47d0a5d7751276962664" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33201 } } peers { permanent_uuid: "02848c8f2aca41a2960fab89cfd71a6a" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41489 } }
I20260812 17:50:00.543191 32390 raft_consensus.cc:374] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:50:00.543263 32390 raft_consensus.cc:729] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1039dbc721ac47d0a5d7751276962664, State: Initialized, Role: FOLLOWER
I20260812 17:50:00.543390 32391 consensus_queue.cc:260] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4147f3f3e4643a2972bece4e7c93de3" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45129 } } peers { permanent_uuid: "1039dbc721ac47d0a5d7751276962664" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33201 } } peers { permanent_uuid: "02848c8f2aca41a2960fab89cfd71a6a" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41489 } }
I20260812 17:50:00.543758 32390 consensus_queue.cc:260] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4147f3f3e4643a2972bece4e7c93de3" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45129 } } peers { permanent_uuid: "1039dbc721ac47d0a5d7751276962664" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33201 } } peers { permanent_uuid: "02848c8f2aca41a2960fab89cfd71a6a" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41489 } }
I20260812 17:50:00.544647 32391 ts_tablet_manager.cc:1428] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 17:50:00.545063 32151 heartbeater.cc:503] Master 127.13.38.62:44589 was elected leader, sending a full tablet report...
I20260812 17:50:00.545137 32390 ts_tablet_manager.cc:1428] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664: Time spent starting tablet: real 0.010s user 0.003s sys 0.003s
I20260812 17:50:00.545660 32366 heartbeater.cc:503] Master 127.13.38.62:44589 was elected leader, sending a full tablet report...
I20260812 17:50:00.607707 32398 raft_consensus.cc:482] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:50:00.607998 32398 raft_consensus.cc:504] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4147f3f3e4643a2972bece4e7c93de3" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45129 } } peers { permanent_uuid: "1039dbc721ac47d0a5d7751276962664" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33201 } } peers { permanent_uuid: "02848c8f2aca41a2960fab89cfd71a6a" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41489 } }
I20260812 17:50:00.611131 32398 leader_election.cc:288] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1039dbc721ac47d0a5d7751276962664 (127.13.38.3:33201), 02848c8f2aca41a2960fab89cfd71a6a (127.13.38.1:41489)
I20260812 17:50:00.617437 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.610454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:40707 (local address 127.13.38.3:33201)
0812 17:50:00.610607 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:00.610612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:00.611588 (+ 976us) server_negotiation.cc:407] Connection header received
0812 17:50:00.611835 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:00.611839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:00.611897 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:00.611952 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:50:00.612431 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.613158 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.616007 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.616656 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.616745 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.616819 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:00.617193 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 17:50:00.617280 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 17:50:00.617921 32404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.609766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:33201 (local address 127.13.38.2:40707)
0812 17:50:00.611449 (+ 1683us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:00.611465 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:00.611684 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:00.612109 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:00.612112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:00.612132 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:50:00.612273 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.612281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.613344 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.613348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.615419 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.615433 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.616926 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.616930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.617033 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.617059 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 17:50:00.617679 (+ 620us) client_negotiation.cc:240] Negotiation successful
0812 17:50:00.617771 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1598,"thread_start_us":296,"threads_started":1}
I20260812 17:50:00.618630 32294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e26c29b70f4af1ba96b25547874ec1" candidate_uuid: "c4147f3f3e4643a2972bece4e7c93de3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1039dbc721ac47d0a5d7751276962664" is_pre_election: true
I20260812 17:50:00.619012 32294 raft_consensus.cc:2454] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4147f3f3e4643a2972bece4e7c93de3 in term 0.
I20260812 17:50:00.619637 31963 leader_election.cc:302] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1039dbc721ac47d0a5d7751276962664, c4147f3f3e4643a2972bece4e7c93de3; no voters:
I20260812 17:50:00.620007 32398 raft_consensus.cc:2781] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:50:00.620126 32398 raft_consensus.cc:482] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:50:00.620218 32398 raft_consensus.cc:3037] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:50:00.622021 32398 raft_consensus.cc:504] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4147f3f3e4643a2972bece4e7c93de3" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45129 } } peers { permanent_uuid: "1039dbc721ac47d0a5d7751276962664" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33201 } } peers { permanent_uuid: "02848c8f2aca41a2960fab89cfd71a6a" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41489 } }
I20260812 17:50:00.622519 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.610837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:43039 (local address 127.13.38.1:41489)
0812 17:50:00.610949 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:00.610954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:00.613649 (+ 2695us) server_negotiation.cc:407] Connection header received
0812 17:50:00.613847 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:00.613850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:00.613902 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:00.613947 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:50:00.614353 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.615073 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.621139 (+ 6066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.621787 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.621861 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.621918 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:00.622306 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 17:50:00.622396 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 17:50:00.623489 32294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e26c29b70f4af1ba96b25547874ec1" candidate_uuid: "c4147f3f3e4643a2972bece4e7c93de3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1039dbc721ac47d0a5d7751276962664"
I20260812 17:50:00.623785 32294 raft_consensus.cc:3037] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:50:00.625754 32294 raft_consensus.cc:2454] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4147f3f3e4643a2972bece4e7c93de3 in term 1.
I20260812 17:50:00.626085 32398 leader_election.cc:288] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [CANDIDATE]: Term 1 election: Requested vote from peers 1039dbc721ac47d0a5d7751276962664 (127.13.38.3:33201), 02848c8f2aca41a2960fab89cfd71a6a (127.13.38.1:41489)
I20260812 17:50:00.626240 31963 leader_election.cc:302] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1039dbc721ac47d0a5d7751276962664, c4147f3f3e4643a2972bece4e7c93de3; no voters:
W20260812 17:50:00.626415 31927 tablet.cc:2369] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:50:00.626444 32398 raft_consensus.cc:2781] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:50:00.626838 32398 raft_consensus.cc:686] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 1 LEADER]: Becoming Leader. State: Replica: c4147f3f3e4643a2972bece4e7c93de3, State: Running, Role: LEADER
I20260812 17:50:00.627326 32398 consensus_queue.cc:237] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4147f3f3e4643a2972bece4e7c93de3" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45129 } } peers { permanent_uuid: "1039dbc721ac47d0a5d7751276962664" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33201 } } peers { permanent_uuid: "02848c8f2aca41a2960fab89cfd71a6a" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41489 } }
I20260812 17:50:00.631472 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.610727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:41489 (local address 127.13.38.2:43039)
0812 17:50:00.613534 (+ 2807us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:00.613550 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:00.613729 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:00.614072 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:00.614075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:00.614089 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:50:00.614222 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.614227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.619672 (+ 5445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.619675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.620946 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:00.620959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.622003 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.622006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:00.622085 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.622113 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 17:50:00.622216 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 17:50:00.631312 (+ 9096us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2712,"thread_start_us":265,"threads_started":1}
I20260812 17:50:00.632088 31568 catalog_manager.cc:5495] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4147f3f3e4643a2972bece4e7c93de3 (127.13.38.2). New cstate: current_term: 1 leader_uuid: "c4147f3f3e4643a2972bece4e7c93de3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4147f3f3e4643a2972bece4e7c93de3" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 45129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1039dbc721ac47d0a5d7751276962664" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02848c8f2aca41a2960fab89cfd71a6a" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41489 } health_report { overall_health: UNKNOWN } } }
I20260812 17:50:00.633466 31854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e26c29b70f4af1ba96b25547874ec1" candidate_uuid: "c4147f3f3e4643a2972bece4e7c93de3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02848c8f2aca41a2960fab89cfd71a6a" is_pre_election: true
I20260812 17:50:00.633771 31854 raft_consensus.cc:2454] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4147f3f3e4643a2972bece4e7c93de3 in term 0.
I20260812 17:50:00.634184 31855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e26c29b70f4af1ba96b25547874ec1" candidate_uuid: "c4147f3f3e4643a2972bece4e7c93de3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02848c8f2aca41a2960fab89cfd71a6a"
I20260812 17:50:00.634292 31855 raft_consensus.cc:3037] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:50:00.635859 31855 raft_consensus.cc:2454] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4147f3f3e4643a2972bece4e7c93de3 in term 1.
W20260812 17:50:00.646107 32367 tablet.cc:2369] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 17:50:00 dist-test-slave-sb2z krb5kdc[30802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786556999, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:00.658546 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.653359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36620 (local address 127.13.38.2:45129)
0812 17:50:00.653499 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:00.653504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:00.653521 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:50:00.653627 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:00.653631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:00.653702 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:00.653835 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:50:00.654271 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.655116 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.656305 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:00.656653 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:00.656759 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:00.657197 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:50:00.657776 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:50:00.657893 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:00.658340 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 17:50:00.658404 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 17:50:00.666400 31855 raft_consensus.cc:1261] T c2e26c29b70f4af1ba96b25547874ec1 P 02848c8f2aca41a2960fab89cfd71a6a [term 1 FOLLOWER]: Refusing update from remote peer c4147f3f3e4643a2972bece4e7c93de3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:50:00.667511 32294 raft_consensus.cc:1261] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664 [term 1 FOLLOWER]: Refusing update from remote peer c4147f3f3e4643a2972bece4e7c93de3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:50:00.668473 32398 consensus_queue.cc:1035] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1039dbc721ac47d0a5d7751276962664" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:50:00.668771 32408 consensus_queue.cc:1035] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02848c8f2aca41a2960fab89cfd71a6a" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 41489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:50:00.684489 32411 mvcc.cc:205] Tried to move back new op lower bound from 7317737474719936512 to 7317737474573778944. Current Snapshot: MvccSnapshot[applied={T|T < 7317737474719936512}]
I20260812 17:50:00.696167 31567 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:50:00.696374 31567 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:50:00.699255 31567 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 273932c455b844428e43b30713dd4526: Sending DeleteTablet for 3 replicas of tablet c2e26c29b70f4af1ba96b25547874ec1
I20260812 17:50:00.700315 32046 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e26c29b70f4af1ba96b25547874ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:50:00 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:36618
I20260812 17:50:00.700299 32269 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e26c29b70f4af1ba96b25547874ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:50:00 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:38374
I20260812 17:50:00.700999 32428 tablet_replica.cc:330] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664: stopping tablet replica
I20260812 17:50:00.701376 32428 raft_consensus.cc:2229] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:50:00.701593 31815 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e26c29b70f4af1ba96b25547874ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:50:00 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:46152
I20260812 17:50:00.701740 32428 raft_consensus.cc:2258] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:50:00.702311 32425 tablet_replica.cc:330] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3: stopping tablet replica
I20260812 17:50:00.702632 32425 raft_consensus.cc:2229] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:50:00.702908 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 31713
I20260812 17:50:00.703045 32425 raft_consensus.cc:2258] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:50:00.704464 32428 ts_tablet_manager.cc:1905] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:50:00.707598 32428 ts_tablet_manager.cc:1918] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:50:00.707717 32428 log.cc:1196] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/c2e26c29b70f4af1ba96b25547874ec1
I20260812 17:50:00.708091 32428 ts_tablet_manager.cc:1939] T c2e26c29b70f4af1ba96b25547874ec1 P 1039dbc721ac47d0a5d7751276962664: Deleting consensus metadata
I20260812 17:50:00.713419 31549 catalog_manager.cc:4841] TS 1039dbc721ac47d0a5d7751276962664 (127.13.38.3:33201): tablet c2e26c29b70f4af1ba96b25547874ec1 (table test-table [id=035f4b137ee64f0b9592724ad02c8d7f]) successfully deleted
I20260812 17:50:00.717283 32425 ts_tablet_manager.cc:1905] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:50:00.723762 32425 ts_tablet_manager.cc:1918] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:50:00.723907 32425 log.cc:1196] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/c2e26c29b70f4af1ba96b25547874ec1
I20260812 17:50:00.724485 32425 ts_tablet_manager.cc:1939] T c2e26c29b70f4af1ba96b25547874ec1 P c4147f3f3e4643a2972bece4e7c93de3: Deleting consensus metadata
I20260812 17:50:00.726030 31551 catalog_manager.cc:4841] TS c4147f3f3e4643a2972bece4e7c93de3 (127.13.38.2:45129): tablet c2e26c29b70f4af1ba96b25547874ec1 (table test-table [id=035f4b137ee64f0b9592724ad02c8d7f]) successfully deleted
W20260812 17:50:00.730419 31548 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:41489 (error 108)
I20260812 17:50:00.731614 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 31940
I20260812 17:50:00.738186 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:00.737742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:41489 (local address 127.0.0.1:46162)
0812 17:50:00.737986 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:00.738073 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:41489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260812 17:50:00.738435 31548 catalog_manager.cc:4584] TS 02848c8f2aca41a2960fab89cfd71a6a (127.13.38.1:41489): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c2e26c29b70f4af1ba96b25547874ec1: Network error: Client connection negotiation failed: client connection to 127.13.38.1:41489: connect: Connection refused (error 111)
I20260812 17:50:00.747762 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 32163
I20260812 17:50:00.767840 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 31517
I20260812 17:50:00.786545 13464 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 17:50:01.136711 13464 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 17:50:01.136873 13464 mini_ranger.cc:66] Stopping Ranger...
I20260812 17:50:01.493199 13464 mini_ranger.cc:68] Stopped Ranger
2026-08-12 17:50:01.493 UTC [30852] LOG: received smart shutdown request
2026-08-12 17:50:01.498 UTC [30852] LOG: background worker "logical replication launcher" (PID 30867) exited with exit code 1
2026-08-12 17:50:01.498 UTC [30857] LOG: shutting down
2026-08-12 17:50:01.722 UTC [30852] LOG: database system is shut down
2026-08-12T17:50:01Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52723 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:50:01 dist-test-slave-sb2z krb5kdc[32445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:50:01 dist-test-slave-sb2z krb5kdc[32445](info): set up 2 sockets
Aug 12 17:50:01 dist-test-slave-sb2z krb5kdc[32445](info): commencing operation
krb5kdc: starting...
W20260812 17:50:03.115744 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:50:03 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:50:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:50:03Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.38.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.38.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/rangeradmin_127.13.38.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/rangeradmin_127.13.38.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.38.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.38.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/rangerlookup_127.13.38.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/rangerlookup_127.13.38.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.38.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.38.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/HTTP_127.13.38.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/HTTP_127.13.38.20@KRBTEST.COM.keytab.
I20260812 17:50:03.312219 13464 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskuCZvAR/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/postgres -l logfile start
2026-08-12 17:50:06.062 UTC [32482] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 17:50:06.062 UTC [32482] LOG: listening on IPv4 address "127.13.38.20", port 45367
2026-08-12 17:50:06.066 UTC [32482] LOG: listening on Unix socket "/tmp/.s.PGSQL.45367"
2026-08-12 17:50:06.089 UTC [32486] LOG: database system was shut down at 2026-08-12 17:50:04 UTC
2026-08-12 17:50:06.093 UTC [32482] LOG: database system is ready to accept connections
I20260812 17:50:07.306248 13464 mini_postgres.cc:96] Postgres bound to 45367
2026-08-12 17:50:07.314 UTC [32498] FATAL: database "slave" does not exist
127.13.38.20:45367 - accepting connections
I20260812 17:50:07.314883 13464 mini_ranger.cc:161] Starting Ranger...
I20260812 17:50:07.333873 13464 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 17:50:07.896818 13464 mini_ranger.cc:87] Created ranger Postgres database
I20260812 17:50:07.896922 13464 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-admin
2026-08-12 17:50:08,198 [I] DB FLAVOR :POSTGRES
2026-08-12 17:50:08,199 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 17:50:08,199 [I] Checking connection..
2026-08-12 17:50:08,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 17:50:08,459 [I] Checking connection passed.
2026-08-12 17:50:08,459 [I] --------- Verifying version history table ---------
2026-08-12 17:50:08,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 17:50:08,735 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 17:50:08,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 17:50:08,988 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 17:50:08,988 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 17:50:08,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 17:50:09.260 UTC [32602] WARNING: there is no transaction in progress
2026-08-12 17:50:09,271 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 17:50:09,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 17:50:09,546 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 17:50:09,547 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 17:50:09,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 17:50:09,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 17:50:10,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sb2z.c.gcp-upstream.internal','N') ;"
2026-08-12 17:50:10,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 17:50:10,639 [I] Table x_portal_user does not exist in database ranger
2026-08-12 17:50:10,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 17:50:10,921 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 17:50:10,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 17:50:11,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 17:50:11,450 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 17:50:11,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 17:50:12.483 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.502 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.521 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.538 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.555 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.575 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.635 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.644 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.661 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.673 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.685 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.700 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.711 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.719 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.727 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:12.736 UTC [303] WARNING: there is no transaction in progress
2026-08-12 17:50:13,480 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 17:50:13,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 17:50:13,807 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 17:50:13,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 17:50:14,052 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 17:50:14,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 17:50:14,303 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 17:50:14,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 17:50:14,572 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 17:50:14,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sb2z.c.gcp-upstream.internal';"
2026-08-12 17:50:15,095 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 17:50:15,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 17:50:15,355 [I] DB_PATCHES have already been applied
I20260812 17:50:15.361569 13464 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-admin:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskuCZvAR/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 17:50:15.361642 13464 mini_ranger.cc:193] Using host: 127.13.38.20
I20260812 17:50:15.366266 13464 mini_ranger.cc:227] Ranger admin URL: http://127.13.38.20:57091
Aug 12, 2026 5:50:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 5:50:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 5:50:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 5:50:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 5:50:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 5:50:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 5:50:16 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 17:50:16.805187 13464 mini_ranger.cc:160] Time spent starting Ranger: real 9.490s user 0.004s sys 0.009s
Aug 12, 2026 5:50:38 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 5:50:38 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 12, 2026 5:50:38 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 5:50:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 5:50:38 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 5:50:39 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 17:50:39.946098 13464 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.38.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.38.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/rangerkms_127.13.38.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.38.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/rangerkms_127.13.38.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.38.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.38.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.38.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/HTTP_127.13.38.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.38.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/HTTP_127.13.38.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 17:50:40 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557040, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 17:50:40.048534 13464 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 17:50:40.060504 13464 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 17:50:41.045698 13464 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 17:50:41.045797 13464 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-kms
I20260812 17:50:41.045819 13464 mini_ranger_kms.cc:227] Using postgres at 127.13.38.20:45367
2026-08-12 17:50:41,251 [I] DB FLAVOR :POSTGRES
2026-08-12 17:50:41,252 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 17:50:41,252 [I] Checking connection
2026-08-12 17:50:41,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 17:50:41,525 [I] connection success
2026-08-12 17:50:41,526 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 17:50:41,526 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 17:50:41,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 17:50:41,809 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 17:50:41,809 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 17:50:41,809 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 17:50:41,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.38.20:45367/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 17:50:42,111 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 17:50:42.683153 13464 mini_ranger_kms.cc:310] Created kms service
I20260812 17:50:42.856019 13464 mini_ranger_kms.cc:326] Created kudu user
I20260812 17:50:42.902936 13464 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 17:50:42 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557040, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:43.356590 13464 mini_ranger_kms.cc:384] Added ranger policy
I20260812 17:50:43.356891 13464 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-kms:/tmp/dist-test-taskuCZvAR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskuCZvAR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 17:50:43.356920 13464 mini_ranger_kms.cc:240] Using host: 127.13.38.20
I20260812 17:50:43.361939 13464 mini_ranger_kms.cc:276] Ranger KMS PID: 587
I20260812 17:50:43.362044 13464 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.38.20:57929
Aug 12, 2026 5:50:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 5:50:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 5:50:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 5:50:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 5:50:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 5:50:44 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 17:50:45 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786557045, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.38.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:46.171643 13464 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.123s user 0.009s sys 0.181s
I20260812 17:50:46.171780 13464 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 17:50:46.171849 13464 mini_ranger_kms.cc:401] 127.13.38.20:57929/kms/v1/keys
Aug 12 17:50:46 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557045, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.38.20@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
Aug 12 17:50:46 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557045, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.38.20@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
Aug 12 17:50:49 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:50:49.056093 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:35119
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:35119
--ranger_config_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.38.20:57929/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:50:49.196700 656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:50:49.197033 656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:50:49.197114 656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:50:49.201406 656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:50:49.201508 656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:50:49.201548 656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:50:49.201601 656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:50:49.201639 656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:50:49.207438 656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.38.20:57929/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:35119
--ranger_config_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:35119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:50:49.208694 656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:50:49.209848 656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:50:49.219053 664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:49.219194 656 server_base.cc:1034] running on GCE node
W20260812 17:50:49.219448 668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:49.219484 665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:49.220245 656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:50:49.221532 656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:50:49.222720 656 hybrid_clock.cc:648] HybridClock initialized: now 1786557049222695 us; error 51 us; skew 500 ppm
Aug 12 17:50:49 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557049, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:49.225814 656 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:50:49.226948 656 webserver.cc:460] Webserver started at http://127.13.38.62:38581/ using document root <none> and password file <none>
I20260812 17:50:49.227607 656 fs_manager.cc:359] Metadata directory not provided
I20260812 17:50:49.227672 656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:50:49.227910 656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 17:50:49 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557049, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:49.279688 656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad"
format_stamp: "Formatted at 2026-08-12 17:50:49 on dist-test-slave-sb2z"
server_key: "8aa117ed0c2088f98b15a34adfd3c598"
server_key_iv: "8f8e82830bd84dded07077f68ab9ad81"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:49.280453 656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad"
format_stamp: "Formatted at 2026-08-12 17:50:49 on dist-test-slave-sb2z"
server_key: "8aa117ed0c2088f98b15a34adfd3c598"
server_key_iv: "8f8e82830bd84dded07077f68ab9ad81"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:49.284786 656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:50:49.300107 677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:49.301681 656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:50:49.301868 656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad"
format_stamp: "Formatted at 2026-08-12 17:50:49 on dist-test-slave-sb2z"
server_key: "8aa117ed0c2088f98b15a34adfd3c598"
server_key_iv: "8f8e82830bd84dded07077f68ab9ad81"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:49.302019 656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:50:49.317345 656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:50:49.321744 656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:50:49.321995 656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:50:49.331517 656 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:35119
I20260812 17:50:49.331521 766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:35119 every 8 connection(s)
I20260812 17:50:49.332790 656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:50:49.335840 769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:50:49.337026 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 656
I20260812 17:50:49.337158 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:50:49.337492 13464 external_mini_cluster.cc:1428] Setting key a08b3dc7260aa2d3a13f8960f5f9efb2
I20260812 17:50:49.344291 769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad: Bootstrap starting.
I20260812 17:50:49.346952 769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad: Neither blocks nor log segments found. Creating new log.
I20260812 17:50:49.347980 769 log.cc:824] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad: Log is configured to *not* fsync() on all Append() calls
Aug 12 17:50:49 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:50:49.350582 769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad: No bootstrap required, opened a new log
I20260812 17:50:49.354678 769 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35119 } }
I20260812 17:50:49.354959 769 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:50:49.355028 769 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3449a937f5c4ed9b0a7e861e69bf0ad, State: Initialized, Role: FOLLOWER
I20260812 17:50:49.355573 769 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35119 } }
I20260812 17:50:49.355736 769 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:50:49.355803 769 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:50:49.355908 769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:50:49.356897 769 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35119 } }
I20260812 17:50:49.357296 769 leader_election.cc:302] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3449a937f5c4ed9b0a7e861e69bf0ad; no voters:
I20260812 17:50:49.357455 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:49.338955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37066 (local address 127.13.38.62:35119)
0812 17:50:49.339355 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:49.339375 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:49.339420 (+ 45us) server_negotiation.cc:407] Connection header received
0812 17:50:49.341388 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:49.341416 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:49.341862 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:49.342281 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:49.343250 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:49.344711 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:49.345955 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:49.346625 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:49.346807 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:49.349963 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:49.349971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:49.349991 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:49.350025 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:49.353186 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:49.353811 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:49.353819 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:49.353826 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:49.353985 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:49.354348 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:49.354352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:49.354354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:49.354928 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:49.355142 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:49.356083 (+ 941us) server_negotiation.cc:299] Negotiation successful
0812 17:50:49.356398 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":133,"threads_started":1}
I20260812 17:50:49.357733 769 leader_election.cc:288] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:50:49.358136 776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:50:49.358348 776 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 1 LEADER]: Becoming Leader. State: Replica: f3449a937f5c4ed9b0a7e861e69bf0ad, State: Running, Role: LEADER
I20260812 17:50:49.358668 776 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35119 } }
I20260812 17:50:49.359073 769 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:50:49.360029 776 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35119 } } }
I20260812 17:50:49.360139 776 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [sys.catalog]: This master's current role is: LEADER
I20260812 17:50:49.360333 776 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3449a937f5c4ed9b0a7e861e69bf0ad. Latest consensus state: current_term: 1 leader_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35119 } } }
I20260812 17:50:49.360409 776 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [sys.catalog]: This master's current role is: LEADER
I20260812 17:50:49.361497 769 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 17:50:49.367422 787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:50:49.371359 787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:50:49.380358 787 catalog_manager.cc:1347] Generated new cluster ID: 66c4f35c796b49259ff2840a818e1bcc
I20260812 17:50:49.380445 787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:50:49.413347 787 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:50:49.414729 787 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:50:49.429911 787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad: Generated new TSK 0
I20260812 17:50:49.430910 787 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:50:50.224257 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.38.20:57929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35119
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:50:50.401221 831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:50:50.401634 831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:50:50.401705 831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:50:50.408864 831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:50:50.408960 831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:50:50.409067 831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:50:50.418413 831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.38.20:57929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:50:50.419991 831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:50:50.421464 831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:50:50.435624 842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:50.436197 841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:50.436528 831 server_base.cc:1034] running on GCE node
W20260812 17:50:50.436520 844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:50.437256 831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:50:50.438133 831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:50:50.439518 831 hybrid_clock.cc:648] HybridClock initialized: now 1786557050439479 us; error 74 us; skew 500 ppm
Aug 12 17:50:50 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:50.444823 831 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:50:50.446341 831 webserver.cc:460] Webserver started at http://127.13.38.1:35327/ using document root <none> and password file <none>
I20260812 17:50:50.447283 831 fs_manager.cc:359] Metadata directory not provided
I20260812 17:50:50.447463 831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:50:50.447840 831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 17:50:50 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:50.470273 831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "026617057df24ac384cf7e9e23165ecf"
format_stamp: "Formatted at 2026-08-12 17:50:50 on dist-test-slave-sb2z"
server_key: "fb9bd5661abd52cec0bb596d321229b5"
server_key_iv: "635ef0edb932d9b46252748c45e49b4b"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:50.471318 831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "026617057df24ac384cf7e9e23165ecf"
format_stamp: "Formatted at 2026-08-12 17:50:50 on dist-test-slave-sb2z"
server_key: "fb9bd5661abd52cec0bb596d321229b5"
server_key_iv: "635ef0edb932d9b46252748c45e49b4b"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:50.477805 831 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 17:50:50.522210 854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:50.524225 831 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.003s sys 0.001s
I20260812 17:50:50.524402 831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "026617057df24ac384cf7e9e23165ecf"
format_stamp: "Formatted at 2026-08-12 17:50:50 on dist-test-slave-sb2z"
server_key: "fb9bd5661abd52cec0bb596d321229b5"
server_key_iv: "635ef0edb932d9b46252748c45e49b4b"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:50.524605 831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:50:50.545694 831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:50:50.549837 831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:50:50.550184 831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:50:50.550967 831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:50:50.552106 831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:50:50.552172 831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:50.552238 831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:50:50.552268 831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:50.568176 831 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:45095
I20260812 17:50:50.569439 831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:50:50.570597 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 831
I20260812 17:50:50.570725 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:50:50.571050 13464 external_mini_cluster.cc:1428] Setting key d1b1ff4c309778e4ea9173471838039f
I20260812 17:50:50.570250 1031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:45095 every 8 connection(s)
Aug 12 17:50:50 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:50:50.598124 1035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:50.579092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35119 (local address 127.13.38.1:38297)
0812 17:50:50.580283 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:50.580352 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:50.581537 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:50.582570 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:50.582581 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:50.583022 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:50.583854 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:50.583889 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:50.585270 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:50.585273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:50.586896 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:50.586912 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:50.587896 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:50.587901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:50.588129 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:50.588970 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:50.588994 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:50.591095 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:50.593934 (+ 2839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:50.593943 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:50.593964 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:50.594432 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:50.594840 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:50.594844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:50.594846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:50.595088 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:50.595734 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:50.595742 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:50.596219 (+ 477us) client_negotiation.cc:769] Sending connection context
0812 17:50:50.596476 (+ 257us) client_negotiation.cc:240] Negotiation successful
0812 17:50:50.596912 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":207,"threads_started":1}
I20260812 17:50:50.599035 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:50.580529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:38297 (local address 127.13.38.62:35119)
0812 17:50:50.582065 (+ 1536us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:50.582070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:50.582088 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:50:50.582139 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:50.582142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:50.582197 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:50.584043 (+ 1846us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:50.584169 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:50.585093 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:50.587079 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:50.587476 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:50.587680 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:50.591380 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:50.591384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:50.591387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:50.591420 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:50.593727 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:50.594599 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:50.594604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:50.594606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:50.594715 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:50.595242 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:50.595246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:50.595248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:50.595561 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:50.595668 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:50.598676 (+ 3008us) server_negotiation.cc:299] Negotiation successful
0812 17:50:50.598839 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1442,"thread_start_us":219,"threads_started":1}
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:50:50.601121 1032 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35119
I20260812 17:50:50.601419 1032 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:50.602120 1032 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
I20260812 17:50:50.604125 700 ts_manager.cc:194] Registered new tserver with Master: 026617057df24ac384cf7e9e23165ecf (127.13.38.1:45095)
I20260812 17:50:50.605819 700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:38297
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:50:50.635591 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.38.20:57929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35119
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 17:50:50 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786557050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 17:50:50.835005 1039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:50:50.835445 1039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:50:50.835513 1039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:50:50.842537 1039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:50:50.842651 1039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:50:50.842746 1039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:50:50.850989 1039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.38.20:57929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:50:50.852551 1039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:50:50.853892 1039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:50:50.868269 1052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:50.869055 1051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:50.869833 1039 server_base.cc:1034] running on GCE node
W20260812 17:50:50.870165 1056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:50.870675 1039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:50:50.871583 1039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:50:50.872784 1039 hybrid_clock.cc:648] HybridClock initialized: now 1786557050872713 us; error 91 us; skew 500 ppm
Aug 12 17:50:50 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:50.879964 1039 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:50:50.881268 1039 webserver.cc:460] Webserver started at http://127.13.38.2:38185/ using document root <none> and password file <none>
I20260812 17:50:50.882098 1039 fs_manager.cc:359] Metadata directory not provided
I20260812 17:50:50.882251 1039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:50:50.882548 1039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 17:50:50 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:50.904558 1039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "c5e1107a1b08424596d370aefbbc27ee"
format_stamp: "Formatted at 2026-08-12 17:50:50 on dist-test-slave-sb2z"
server_key: "cc71c400817b63ff668510b7ec9acfd7"
server_key_iv: "174e349478e94b9b8e628834ed5dc12a"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:50.905591 1039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "c5e1107a1b08424596d370aefbbc27ee"
format_stamp: "Formatted at 2026-08-12 17:50:50 on dist-test-slave-sb2z"
server_key: "cc71c400817b63ff668510b7ec9acfd7"
server_key_iv: "174e349478e94b9b8e628834ed5dc12a"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:50.917477 1039 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.004s sys 0.000s
I20260812 17:50:50.941643 1067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:50.943557 1039 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260812 17:50:50.943724 1039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "c5e1107a1b08424596d370aefbbc27ee"
format_stamp: "Formatted at 2026-08-12 17:50:50 on dist-test-slave-sb2z"
server_key: "cc71c400817b63ff668510b7ec9acfd7"
server_key_iv: "174e349478e94b9b8e628834ed5dc12a"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:50.943866 1039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:50:50.961651 1039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:50:50.965785 1039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:50:50.966037 1039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:50:50.966950 1039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:50:50.968474 1039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:50:50.968540 1039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:50.968662 1039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:50:50.968695 1039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:50.982261 1039 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:34469
I20260812 17:50:50.983743 1039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:50:50.984756 1248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:34469 every 8 connection(s)
I20260812 17:50:50.987470 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 1039
I20260812 17:50:50.987660 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:50:50.988027 13464 external_mini_cluster.cc:1428] Setting key e65bee2aab5149d54caf3a9dc6b0e5fd
Aug 12 17:50:50 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:50:51.009536 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:50.989461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:58871 (local address 127.13.38.62:35119)
0812 17:50:50.989800 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:50.989806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:50.991475 (+ 1669us) server_negotiation.cc:407] Connection header received
0812 17:50:50.993176 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:50.993180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:50.993242 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:50.993300 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:50.995496 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:50.996392 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:50.998697 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:50.999078 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:50.999206 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:51.002779 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:51.002783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:51.002787 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:51.002821 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:51.006250 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:51.007140 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:51.007144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:51.007146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:51.007258 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:51.007785 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:51.007789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:51.007791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:51.008075 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:51.008192 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:51.009160 (+ 968us) server_negotiation.cc:299] Negotiation successful
0812 17:50:51.009309 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260812 17:50:51.010530 1254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:50.988078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35119 (local address 127.13.38.2:58871)
0812 17:50:50.991283 (+ 3205us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:50.991356 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:50.992864 (+ 1508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:50.993785 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:50.993796 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:50.994440 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:50.995276 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:50.995300 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:50.996552 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:50.996557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:50.998232 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:50.998246 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:50.999287 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:50.999292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:50.999560 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:51.000388 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:51.000414 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:51.002552 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:51.006400 (+ 3848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:51.006409 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:51.006434 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:51.007005 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:51.007361 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:51.007365 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:51.007367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:51.007654 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:51.008544 (+ 890us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:51.008554 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:51.008871 (+ 317us) client_negotiation.cc:769] Sending connection context
0812 17:50:51.009096 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 17:50:51.009473 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2940,"thread_start_us":191,"threads_started":1}
I20260812 17:50:51.011991 1252 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35119
I20260812 17:50:51.012279 1252 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:51.013031 1252 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
I20260812 17:50:51.014386 700 ts_manager.cc:194] Registered new tserver with Master: c5e1107a1b08424596d370aefbbc27ee (127.13.38.2:34469)
I20260812 17:50:51.015009 700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:58871
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:50:51.043131 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.38.20:57929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35119
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 17:50:51 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
W20260812 17:50:51.228618 1259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:50:51.229029 1259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:50:51.229101 1259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:50:51.235419 1259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:50:51.235606 1259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:50:51.235725 1259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:50:51.244412 1259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.38.20:57929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:35119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:50:51.246196 1259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:50:51.247661 1259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:50:51.266194 1268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:51.266192 1266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:51.266775 1265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:51.267457 1259 server_base.cc:1034] running on GCE node
I20260812 17:50:51.268026 1259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:50:51.268875 1259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:50:51.276257 1259 hybrid_clock.cc:648] HybridClock initialized: now 1786557051276192 us; error 85 us; skew 500 ppm
Aug 12 17:50:51 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557051, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:51.280689 1259 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:50:51.281944 1259 webserver.cc:460] Webserver started at http://127.13.38.3:43409/ using document root <none> and password file <none>
I20260812 17:50:51.282733 1259 fs_manager.cc:359] Metadata directory not provided
I20260812 17:50:51.282821 1259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:50:51.283061 1259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 17:50:51 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557051, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:51.302829 1259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "0130461f10f94f43a966b8bc44d5da95"
format_stamp: "Formatted at 2026-08-12 17:50:51 on dist-test-slave-sb2z"
server_key: "e1e96f5702c06af9d679296e52b47ea4"
server_key_iv: "bfe2074e4f358a3736daedec883fd512"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:51.303766 1259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "0130461f10f94f43a966b8bc44d5da95"
format_stamp: "Formatted at 2026-08-12 17:50:51 on dist-test-slave-sb2z"
server_key: "e1e96f5702c06af9d679296e52b47ea4"
server_key_iv: "bfe2074e4f358a3736daedec883fd512"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:51.310380 1259 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 17:50:51.340817 1275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:51.343173 1259 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260812 17:50:51.343355 1259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "0130461f10f94f43a966b8bc44d5da95"
format_stamp: "Formatted at 2026-08-12 17:50:51 on dist-test-slave-sb2z"
server_key: "e1e96f5702c06af9d679296e52b47ea4"
server_key_iv: "bfe2074e4f358a3736daedec883fd512"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:51.343560 1259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:50:51.391855 1259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:50:51.396792 1259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:50:51.397153 1259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:50:51.398083 1259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:50:51.399389 1259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:50:51.399509 1259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:51.399662 1259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:50:51.399742 1259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:51.422013 1259 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:33089
I20260812 17:50:51.423714 1259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:50:51.424772 1445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:33089 every 8 connection(s)
I20260812 17:50:51.429199 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 1259
I20260812 17:50:51.429396 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:50:51.429888 13464 external_mini_cluster.cc:1428] Setting key cbc3457d28ea40d3fc530344789e548e
Aug 12 17:50:51 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557051, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:50:51.444646 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:51.427860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:41785 (local address 127.13.38.62:35119)
0812 17:50:51.428022 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:51.428028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:51.428594 (+ 566us) server_negotiation.cc:407] Connection header received
0812 17:50:51.430444 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:51.430449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:51.430514 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:51.430579 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:51.432608 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:51.433458 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:51.435384 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:51.435803 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:51.435891 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:51.439463 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:51.439468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:51.439471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:51.439506 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:51.441554 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:51.442359 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:51.442364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:51.442366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:51.442469 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:51.442958 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:51.442961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:51.442964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:51.443247 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:51.443336 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:51.444293 (+ 957us) server_negotiation.cc:299] Negotiation successful
0812 17:50:51.444444 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 17:50:51.446887 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:51.427500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35119 (local address 127.13.38.3:41785)
0812 17:50:51.428372 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:51.428458 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:51.430137 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:51.431092 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:51.431103 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:51.431618 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:51.432413 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:51.432438 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:51.433652 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:51.433656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:51.435217 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:51.435231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:51.436142 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:51.436146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:51.436371 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:51.437191 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:51.437215 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:51.439208 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:51.441741 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:51.441748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:51.441770 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:51.442215 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:51.442601 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:51.442604 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:51.442606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:51.442824 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:51.443436 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:51.443444 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:51.443928 (+ 484us) client_negotiation.cc:769] Sending connection context
0812 17:50:51.445205 (+ 1277us) client_negotiation.cc:240] Negotiation successful
0812 17:50:51.445663 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":200,"threads_started":1}
I20260812 17:50:51.448939 1446 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35119
I20260812 17:50:51.449256 1446 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:51.450039 1446 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
I20260812 17:50:51.451702 700 ts_manager.cc:194] Registered new tserver with Master: 0130461f10f94f43a966b8bc44d5da95 (127.13.38.3:33089)
I20260812 17:50:51.452473 700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:41785
I20260812 17:50:51.457115 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:50:51.458076 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 831
Aug 12 17:50:51 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:51.582464 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 1039
I20260812 17:50:51.607173 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 1259
I20260812 17:50:51.627350 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 656
I20260812 17:50:51.650472 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:35119
--webserver_interface=127.13.38.62
--webserver_port=38581
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:35119
--ranger_config_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.38.20:57929/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.38.20:57929/kms with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:50:51.802917 1452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:50:51.803233 1452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:50:51.803277 1452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:50:51.807778 1452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:50:51.807859 1452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:50:51.807886 1452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:50:51.807904 1452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:50:51.807962 1452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:50:51.814304 1452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.38.20:57929/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:35119
--ranger_config_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:35119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=38581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:50:51.815572 1452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:50:51.817116 1452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:50:51.828438 1463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:51.828440 1461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:51.828438 1466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:51.829810 1452 server_base.cc:1034] running on GCE node
I20260812 17:50:51.830546 1452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:50:51.831977 1452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:50:51.833185 1452 hybrid_clock.cc:648] HybridClock initialized: now 1786557051833153 us; error 62 us; skew 500 ppm
Aug 12 17:50:51 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557051, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:51.836599 1452 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:50:51.837805 1452 webserver.cc:460] Webserver started at http://127.13.38.62:38581/ using document root <none> and password file <none>
I20260812 17:50:51.838418 1452 fs_manager.cc:359] Metadata directory not provided
I20260812 17:50:51.838491 1452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 17:50:51 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557051, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:51.888326 1452 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:50:51.890841 1479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:51.892366 1452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:50:51.892522 1452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad"
format_stamp: "Formatted at 2026-08-12 17:50:49 on dist-test-slave-sb2z"
server_key: "8aa117ed0c2088f98b15a34adfd3c598"
server_key_iv: "8f8e82830bd84dded07077f68ab9ad81"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:51.893139 1452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:50:51.914027 1452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:50:51.917464 1452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:50:51.917990 1452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:50:51.925850 1556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:35119 every 8 connection(s)
I20260812 17:50:51.925848 1452 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:35119
I20260812 17:50:51.927171 1452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:50:51.931906 1558 sys_catalog.cc:263] Verifying existing consensus state
I20260812 17:50:51.933771 1558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad: Bootstrap starting.
I20260812 17:50:51.934828 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 1452
I20260812 17:50:51.935417 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.38.20:57929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:45095
--local_ip_for_outbound_sockets=127.13.38.1
--tserver_master_addrs=127.13.38.62:35119
--webserver_port=35327
--webserver_interface=127.13.38.1
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.38.20:57929/kms with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:50:51.952754 1558 log.cc:824] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad: Log is configured to *not* fsync() on all Append() calls
I20260812 17:50:51.960206 1558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 17:50:51.960681 1558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad: Bootstrap complete.
I20260812 17:50:51.965236 1558 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35119 } }
I20260812 17:50:51.965669 1558 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3449a937f5c4ed9b0a7e861e69bf0ad, State: Initialized, Role: FOLLOWER
I20260812 17:50:51.966203 1558 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35119 } }
I20260812 17:50:51.966329 1558 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:50:51.966434 1558 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:50:51.966554 1558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 1 FOLLOWER]: Advancing to term 2
I20260812 17:50:51.968699 1558 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35119 } }
I20260812 17:50:51.969069 1558 leader_election.cc:302] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3449a937f5c4ed9b0a7e861e69bf0ad; no voters:
I20260812 17:50:51.969434 1558 leader_election.cc:288] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 17:50:51.969529 1564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 2 FOLLOWER]: Leader election won for term 2
I20260812 17:50:51.969766 1564 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [term 2 LEADER]: Becoming Leader. State: Replica: f3449a937f5c4ed9b0a7e861e69bf0ad, State: Running, Role: LEADER
I20260812 17:50:51.970096 1564 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35119 } }
I20260812 17:50:51.970850 1558 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:50:51.971570 1569 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3449a937f5c4ed9b0a7e861e69bf0ad. Latest consensus state: current_term: 2 leader_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35119 } } }
I20260812 17:50:51.972018 1569 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [sys.catalog]: This master's current role is: LEADER
I20260812 17:50:51.971895 1566 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3449a937f5c4ed9b0a7e861e69bf0ad" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 35119 } } }
I20260812 17:50:51.972362 1566 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad [sys.catalog]: This master's current role is: LEADER
I20260812 17:50:51.972668 1576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:50:51.976231 1576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:50:51.976881 1558 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 17:50:51.977095 1576 catalog_manager.cc:1259] Loaded cluster ID: 66c4f35c796b49259ff2840a818e1bcc
I20260812 17:50:51.977200 1576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:50:51.979653 1576 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:50:51.983603 1576 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad: Loaded TSK: 0
I20260812 17:50:51.985623 1576 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 17:50:52.104478 1560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:50:52.104902 1560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:50:52.104993 1560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:50:52.111790 1560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:50:52.111881 1560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:50:52.111972 1560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:50:52.120306 1560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.38.20:57929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:45095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=35327
--tserver_master_addrs=127.13.38.62:35119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:50:52.121913 1560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:50:52.123371 1560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:50:52.140342 1607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:52.140478 1560 server_base.cc:1034] running on GCE node
W20260812 17:50:52.140388 1609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:52.140628 1606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:52.141342 1560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:50:52.142078 1560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:50:52.143308 1560 hybrid_clock.cc:648] HybridClock initialized: now 1786557052143265 us; error 58 us; skew 500 ppm
Aug 12 17:50:52 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557052, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:52.146838 1560 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:50:52.147965 1560 webserver.cc:460] Webserver started at http://127.13.38.1:35327/ using document root <none> and password file <none>
I20260812 17:50:52.148592 1560 fs_manager.cc:359] Metadata directory not provided
I20260812 17:50:52.148717 1560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 17:50:52 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557052, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:52.174921 1560 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260812 17:50:52.180460 1621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:52.182538 1560 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260812 17:50:52.182734 1560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "026617057df24ac384cf7e9e23165ecf"
format_stamp: "Formatted at 2026-08-12 17:50:50 on dist-test-slave-sb2z"
server_key: "fb9bd5661abd52cec0bb596d321229b5"
server_key_iv: "635ef0edb932d9b46252748c45e49b4b"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:52.183586 1560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:50:52.225983 1560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:50:52.230322 1560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:50:52.230585 1560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:50:52.231429 1560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:50:52.232676 1560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:50:52.232753 1560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:52.232825 1560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:50:52.232856 1560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:52.248281 1560 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:45095
I20260812 17:50:52.249797 1560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:50:52.250712 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 1560
I20260812 17:50:52.251259 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.38.20:57929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:34469
--local_ip_for_outbound_sockets=127.13.38.2
--tserver_master_addrs=127.13.38.62:35119
--webserver_port=38185
--webserver_interface=127.13.38.2
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.38.20:57929/kms with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:50:52.251183 1788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:45095 every 8 connection(s)
Aug 12 17:50:52 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557052, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:50:52.278847 1794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:52.254805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35119 (local address 127.13.38.1:39413)
0812 17:50:52.257552 (+ 2747us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:52.257625 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:52.258886 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:52.261479 (+ 2593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:52.261489 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:52.261905 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:52.262631 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:52.262655 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.264826 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.264830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:52.266407 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:52.266422 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.267908 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.267913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:52.268149 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:52.269006 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:52.269029 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:52.270950 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:52.274285 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:52.274293 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:52.274313 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:52.274746 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:52.275240 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:52.275243 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:52.275245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:52.275465 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:52.276528 (+ 1063us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:52.276536 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:52.276905 (+ 369us) client_negotiation.cc:769] Sending connection context
0812 17:50:52.277132 (+ 227us) client_negotiation.cc:240] Negotiation successful
0812 17:50:52.277620 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2444,"thread_start_us":174,"threads_started":1}
I20260812 17:50:52.281392 1795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:52.256698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:39413 (local address 127.13.38.62:35119)
0812 17:50:52.259500 (+ 2802us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:52.259547 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:52.259602 (+ 55us) server_negotiation.cc:407] Connection header received
0812 17:50:52.259940 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:52.259957 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:52.260440 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:52.260836 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:52.262876 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.264640 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.266581 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.267420 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.267643 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:52.271228 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:52.271236 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:52.271257 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:52.271297 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:52.274103 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:52.274920 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:52.274927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:52.274934 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:52.275109 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:52.275634 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:52.275638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:52.275640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:52.276295 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:52.276447 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:52.279910 (+ 3463us) server_negotiation.cc:299] Negotiation successful
0812 17:50:52.280283 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2590,"thread_start_us":114,"threads_started":1}
I20260812 17:50:52.285349 1790 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35119
I20260812 17:50:52.285650 1790 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:52.286303 1790 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
W20260812 17:50:52.289283 1790 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:35119 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 17:50:52.289458 1790 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:52.289683 1790 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
I20260812 17:50:52.290977 1790 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:52.291172 1790 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
W20260812 17:50:52.292162 1790 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 17:50:52.429473 1793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:50:52.429935 1793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:50:52.430027 1793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:50:52.435884 1793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:50:52.435981 1793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:50:52.436074 1793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:50:52.442641 1793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.38.20:57929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:34469
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=38185
--tserver_master_addrs=127.13.38.62:35119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:50:52.443941 1793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:50:52.445092 1793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:50:52.460888 1803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:52.461102 1793 server_base.cc:1034] running on GCE node
W20260812 17:50:52.460902 1805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:52.463757 1801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:52.464334 1793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:50:52.465163 1793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:50:52.466359 1793 hybrid_clock.cc:648] HybridClock initialized: now 1786557052466317 us; error 54 us; skew 500 ppm
Aug 12 17:50:52 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557052, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:52.470006 1793 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:50:52.471283 1793 webserver.cc:460] Webserver started at http://127.13.38.2:38185/ using document root <none> and password file <none>
I20260812 17:50:52.472090 1793 fs_manager.cc:359] Metadata directory not provided
I20260812 17:50:52.472213 1793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 17:50:52 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557052, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:52.493440 1793 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:50:52.496502 1815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:52.498016 1793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:50:52.498209 1793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "c5e1107a1b08424596d370aefbbc27ee"
format_stamp: "Formatted at 2026-08-12 17:50:50 on dist-test-slave-sb2z"
server_key: "cc71c400817b63ff668510b7ec9acfd7"
server_key_iv: "174e349478e94b9b8e628834ed5dc12a"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:52.498819 1793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:50:52.518854 1793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:50:52.524966 1793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:50:52.525297 1793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:50:52.526229 1793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:50:52.527462 1793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:50:52.527619 1793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:52.527745 1793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:50:52.527837 1793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:52.539084 1793 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:34469
I20260812 17:50:52.540715 1984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:34469 every 8 connection(s)
I20260812 17:50:52.540921 1793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:50:52.550396 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 1793
I20260812 17:50:52.550995 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.38.20:57929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:33089
--local_ip_for_outbound_sockets=127.13.38.3
--tserver_master_addrs=127.13.38.62:35119
--webserver_port=43409
--webserver_interface=127.13.38.3
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.38.20:57929/kms with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 17:50:52 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557052, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:50:52.570293 1795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:52.543111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:40667 (local address 127.13.38.62:35119)
0812 17:50:52.543293 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:52.543299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:52.544470 (+ 1171us) server_negotiation.cc:407] Connection header received
0812 17:50:52.545522 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:52.545525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:52.545595 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:52.545677 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:52.546885 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.547425 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.548634 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.548865 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.548982 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:52.562247 (+ 13265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:52.562252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:52.562255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:52.562295 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:52.564357 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:52.565175 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:52.565181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:52.565186 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:52.565294 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:52.565827 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:52.565832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:52.565835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:52.566153 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:52.566258 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:52.569842 (+ 3584us) server_negotiation.cc:299] Negotiation successful
0812 17:50:52.570029 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 17:50:52.572417 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:52.543689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35119 (local address 127.13.38.2:40667)
0812 17:50:52.544295 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:52.544350 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:52.545294 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:52.545887 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:52.545895 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:52.546218 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:52.546725 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:52.546741 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.547597 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.547606 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:52.548484 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:52.548494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.549032 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.549036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:52.549219 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:52.549869 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:52.549886 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:52.561831 (+ 11945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:52.564558 (+ 2727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:52.564568 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:52.564591 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:52.565049 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:52.565429 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:52.565434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:52.565437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:52.565685 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:52.566346 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:52.566356 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:52.566810 (+ 454us) client_negotiation.cc:769] Sending connection context
0812 17:50:52.570801 (+ 3991us) client_negotiation.cc:240] Negotiation successful
0812 17:50:52.571238 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":168,"threads_started":1}
I20260812 17:50:52.574165 1985 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35119
I20260812 17:50:52.574429 1985 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:52.575034 1985 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
W20260812 17:50:52.576879 1985 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:35119 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 17:50:52.577015 1985 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:52.577208 1985 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
I20260812 17:50:52.578063 1985 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:52.578244 1985 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
W20260812 17:50:52.579136 1985 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 17:50:52.723275 1992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:50:52.723768 1992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:50:52.723872 1992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:50:52.730172 1992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:50:52.730293 1992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:50:52.730423 1992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:50:52.737082 1992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.38.20:57929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:33089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=43409
--tserver_master_addrs=127.13.38.62:35119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:50:52.738320 1992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:50:52.739737 1992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:50:52.751830 2001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:52.751838 2007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:52.752631 1999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:52.753608 1992 server_base.cc:1034] running on GCE node
I20260812 17:50:52.754175 1992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:50:52.755085 1992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:50:52.758391 1992 hybrid_clock.cc:648] HybridClock initialized: now 1786557052758354 us; error 46 us; skew 500 ppm
Aug 12 17:50:52 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557052, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:52.762657 1992 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:50:52.764089 1992 webserver.cc:460] Webserver started at http://127.13.38.3:43409/ using document root <none> and password file <none>
I20260812 17:50:52.764971 1992 fs_manager.cc:359] Metadata directory not provided
I20260812 17:50:52.765156 1992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 17:50:52 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557052, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:52.789186 1992 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.004s sys 0.000s
I20260812 17:50:52.796454 2021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:52.798573 1992 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.000s
I20260812 17:50:52.798746 1992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "0130461f10f94f43a966b8bc44d5da95"
format_stamp: "Formatted at 2026-08-12 17:50:51 on dist-test-slave-sb2z"
server_key: "e1e96f5702c06af9d679296e52b47ea4"
server_key_iv: "bfe2074e4f358a3736daedec883fd512"
server_key_version: "kuduclusterkey@0"
I20260812 17:50:52.799624 1992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:50:52.823618 1992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:50:52.827696 1992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:50:52.827919 1992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:50:52.828706 1992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:50:52.829830 1992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:50:52.829900 1992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:52.829959 1992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:50:52.829993 1992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:52.842306 1992 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:33089
I20260812 17:50:52.843874 1992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:50:52.844575 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 1992
I20260812 17:50:52.844830 2193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:33089 every 8 connection(s)
I20260812 17:50:52.859275 1795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:52.845806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43220 (local address 127.13.38.62:35119)
0812 17:50:52.845947 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:52.845952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:52.849124 (+ 3172us) server_negotiation.cc:407] Connection header received
0812 17:50:52.849189 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:52.849193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:52.849253 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:52.849343 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:52.850513 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.851291 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.852607 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.852950 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.853051 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:52.855086 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:52.855091 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:52.855094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:52.855126 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:52.856979 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:52.857627 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:52.857630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:52.857633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:52.857727 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:52.858087 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:52.858091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:52.858093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:52.858317 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:52.858512 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:52.858969 (+ 457us) server_negotiation.cc:299] Negotiation successful
0812 17:50:52.859098 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 17:50:52 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557052, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:50:52.867038 2198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:52.848355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:34161 (local address 127.13.38.62:35119)
0812 17:50:52.854305 (+ 5950us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:52.854310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:52.854326 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:50:52.854367 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:52.854370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:52.854419 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:52.854515 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:52.856211 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.856857 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.858706 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.859056 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.859156 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:52.862221 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:52.862225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:52.862228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:52.862255 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:52.864245 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:52.864975 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:52.864979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:52.864981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:52.865080 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:52.865541 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:52.865545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:52.865547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:52.865807 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:52.865922 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:52.866682 (+ 760us) server_negotiation.cc:299] Negotiation successful
0812 17:50:52.866826 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5860,"thread_start_us":112,"threads_started":1}
I20260812 17:50:52.868283 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:52.848601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:35119 (local address 127.13.38.3:34161)
0812 17:50:52.849840 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:52.849899 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:52.851005 (+ 1106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:52.854852 (+ 3847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:52.854863 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:52.855238 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:52.856020 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:52.856047 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.856996 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.857000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:52.858556 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:52.858568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:52.859340 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:52.859343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:52.859615 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:52.860277 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:52.860299 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:52.862029 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:52.864383 (+ 2354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:52.864390 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:52.864410 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:52.864833 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:52.865194 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:52.865197 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:52.865199 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:52.865413 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:52.865954 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:52.865963 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:52.866360 (+ 397us) client_negotiation.cc:769] Sending connection context
0812 17:50:52.866611 (+ 251us) client_negotiation.cc:240] Negotiation successful
0812 17:50:52.866979 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1012,"thread_start_us":160,"threads_started":1}
I20260812 17:50:52.869611 2194 heartbeater.cc:347] Connected to a master server at 127.13.38.62:35119
I20260812 17:50:52.869925 2194 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:52.870636 2194 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
W20260812 17:50:52.872833 2194 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:35119 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 17:50:52.872992 2194 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:52.873222 2194 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
I20260812 17:50:52.874066 2194 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:52.874264 2194 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
W20260812 17:50:52.875125 2194 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 17:50:53.292409 1790 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:53.292788 1790 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
I20260812 17:50:53.294246 1506 ts_manager.cc:194] Registered new tserver with Master: 026617057df24ac384cf7e9e23165ecf (127.13.38.1:45095)
I20260812 17:50:53.296356 1506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:39413
Aug 12 17:50:53 dist-test-slave-sb2z krb5kdc[32445](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786557053, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:53.579349 1985 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:53.579811 1985 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
I20260812 17:50:53.580614 1506 ts_manager.cc:194] Registered new tserver with Master: c5e1107a1b08424596d370aefbbc27ee (127.13.38.2:34469)
I20260812 17:50:53.581424 1506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:40667
Aug 12 17:50:53 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557053, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
I20260812 17:50:53.875378 2194 heartbeater.cc:464] Registering TS with master...
I20260812 17:50:53.875813 2194 heartbeater.cc:511] Master 127.13.38.62:35119 requested a full tablet report, sending...
I20260812 17:50:53.876720 1506 ts_manager.cc:194] Registered new tserver with Master: 0130461f10f94f43a966b8bc44d5da95 (127.13.38.3:33089)
I20260812 17:50:53.877494 1506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:34161
I20260812 17:50:53.884855 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:50:53.897558 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:53.886599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43234 (local address 127.13.38.62:35119)
0812 17:50:53.887316 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:53.887322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:53.887340 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:50:53.887387 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:53.887390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:53.887438 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:53.887571 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:53.888482 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.889421 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.890766 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.891133 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.891269 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:53.893186 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:53.893190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:53.893193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:53.893224 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:53.895236 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:53.895840 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:53.895844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:53.895846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:53.895965 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:53.896309 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:53.896313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:53.896314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:53.896577 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:53.896754 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:53.897210 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 17:50:53.897356 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":86,"threads_started":1}
I20260812 17:50:53.902244 1506 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:50:53.907395 1506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:50:53.933394 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:53.922171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:33089 (local address 127.0.0.1:55878)
0812 17:50:53.924355 (+ 2184us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:53.924413 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:53.924585 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:53.926155 (+ 1570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:53.926162 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:53.926186 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:50:53.926339 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:53.926346 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.928058 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.928062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:53.930924 (+ 2862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:53.930935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.932822 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.932826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:53.932946 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:53.933003 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 17:50:53.933098 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 17:50:53.933241 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1926,"mutex_wait_us":1414,"thread_start_us":99,"threads_started":1}
I20260812 17:50:53.935045 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:53.923030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55878 (local address 127.13.38.3:33089)
0812 17:50:53.925600 (+ 2570us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:53.925610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:53.925638 (+ 28us) server_negotiation.cc:407] Connection header received
0812 17:50:53.925720 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:53.925726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:53.925971 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:53.926521 (+ 550us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:50:53.926691 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.927913 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.931719 (+ 3806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.932662 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.934550 (+ 1888us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:53.934735 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:53.934789 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 17:50:53.934887 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2447,"thread_start_us":87,"threads_started":1}
I20260812 17:50:53.937587 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:53.924559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45095 (local address 127.0.0.1:50104)
0812 17:50:53.924837 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:53.924849 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:53.924925 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:53.926907 (+ 1982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:53.926910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:53.926926 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:50:53.927027 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:53.927032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.928623 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.928626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:53.931609 (+ 2983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:53.931617 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.937207 (+ 5590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.937211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:53.937305 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:53.937332 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 17:50:53.937379 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 17:50:53.937449 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":1,"thread_start_us":109,"threads_started":1}
I20260812 17:50:53.938544 2086 tablet_service.cc:1467] Processing CreateTablet for tablet 25b77df7e76e486caa3a117e0c13441c (DEFAULT_TABLE table=test-table [id=0fbe8b2fb0814bfb9b655e5b4b283524]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:50:53.939108 2232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:53.924994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50104 (local address 127.13.38.1:45095)
0812 17:50:53.925724 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:53.925732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:53.925759 (+ 27us) server_negotiation.cc:407] Connection header received
0812 17:50:53.925840 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:53.925845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:53.926047 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:53.927168 (+ 1121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:50:53.927315 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.928491 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.936146 (+ 7655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.937063 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.937869 (+ 806us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:53.938050 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:53.938103 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 17:50:53.938198 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":73,"threads_started":1}
I20260812 17:50:53.939812 2086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25b77df7e76e486caa3a117e0c13441c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:50:53.942500 1698 tablet_service.cc:1467] Processing CreateTablet for tablet 25b77df7e76e486caa3a117e0c13441c (DEFAULT_TABLE table=test-table [id=0fbe8b2fb0814bfb9b655e5b4b283524]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:50:53.943693 1698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25b77df7e76e486caa3a117e0c13441c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:50:53.945394 2234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:53.925523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:34469 (local address 127.0.0.1:48304)
0812 17:50:53.928846 (+ 3323us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:53.928861 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:53.928953 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:53.939958 (+ 11005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:53.939961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:53.939993 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:50:53.940094 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:53.940100 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.941739 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.941743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:53.942863 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:53.942876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.944940 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.944944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:53.945050 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:53.945081 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 17:50:53.945139 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 17:50:53.945220 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3254,"thread_start_us":81,"threads_started":1}
I20260812 17:50:53.946142 2235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:53.925703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48304 (local address 127.13.38.2:34469)
0812 17:50:53.939392 (+ 13689us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:53.939401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:53.939431 (+ 30us) server_negotiation.cc:407] Connection header received
0812 17:50:53.939546 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:53.939564 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:53.939797 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:53.940266 (+ 469us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:50:53.940439 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.941597 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.943049 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:53.943850 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:53.945675 (+ 1825us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:53.945839 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:53.945893 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 17:50:53.946000 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13560,"thread_start_us":83,"threads_started":1}
I20260812 17:50:53.949350 1889 tablet_service.cc:1467] Processing CreateTablet for tablet 25b77df7e76e486caa3a117e0c13441c (DEFAULT_TABLE table=test-table [id=0fbe8b2fb0814bfb9b655e5b4b283524]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:50:53.950631 1889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25b77df7e76e486caa3a117e0c13441c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:50:53.954630 2236 tablet_bootstrap.cc:492] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf: Bootstrap starting.
I20260812 17:50:53.957398 2236 tablet_bootstrap.cc:654] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf: Neither blocks nor log segments found. Creating new log.
I20260812 17:50:53.958426 2236 log.cc:824] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf: Log is configured to *not* fsync() on all Append() calls
I20260812 17:50:53.961668 2236 tablet_bootstrap.cc:492] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf: No bootstrap required, opened a new log
I20260812 17:50:53.961915 2236 ts_tablet_manager.cc:1397] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 17:50:53.966568 2239 tablet_bootstrap.cc:492] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee: Bootstrap starting.
I20260812 17:50:53.966470 2236 raft_consensus.cc:348] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0130461f10f94f43a966b8bc44d5da95" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33089 } } peers { permanent_uuid: "026617057df24ac384cf7e9e23165ecf" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45095 } } peers { permanent_uuid: "c5e1107a1b08424596d370aefbbc27ee" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34469 } }
I20260812 17:50:53.967010 2238 tablet_bootstrap.cc:492] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95: Bootstrap starting.
I20260812 17:50:53.967188 2236 raft_consensus.cc:374] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:50:53.967298 2236 raft_consensus.cc:729] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 026617057df24ac384cf7e9e23165ecf, State: Initialized, Role: FOLLOWER
I20260812 17:50:53.967922 2236 consensus_queue.cc:260] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0130461f10f94f43a966b8bc44d5da95" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33089 } } peers { permanent_uuid: "026617057df24ac384cf7e9e23165ecf" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45095 } } peers { permanent_uuid: "c5e1107a1b08424596d370aefbbc27ee" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34469 } }
I20260812 17:50:53.969347 2236 ts_tablet_manager.cc:1428] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 17:50:53.969818 1790 heartbeater.cc:503] Master 127.13.38.62:35119 was elected leader, sending a full tablet report...
I20260812 17:50:53.974362 2238 tablet_bootstrap.cc:654] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95: Neither blocks nor log segments found. Creating new log.
I20260812 17:50:53.975193 2239 tablet_bootstrap.cc:654] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee: Neither blocks nor log segments found. Creating new log.
I20260812 17:50:53.976183 2238 log.cc:824] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95: Log is configured to *not* fsync() on all Append() calls
I20260812 17:50:53.976565 2239 log.cc:824] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee: Log is configured to *not* fsync() on all Append() calls
I20260812 17:50:53.978883 2239 tablet_bootstrap.cc:492] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee: No bootstrap required, opened a new log
I20260812 17:50:53.979179 2239 ts_tablet_manager.cc:1397] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260812 17:50:53.979386 2238 tablet_bootstrap.cc:492] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95: No bootstrap required, opened a new log
I20260812 17:50:53.979633 2238 ts_tablet_manager.cc:1397] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260812 17:50:53.984405 2239 raft_consensus.cc:348] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0130461f10f94f43a966b8bc44d5da95" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33089 } } peers { permanent_uuid: "026617057df24ac384cf7e9e23165ecf" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45095 } } peers { permanent_uuid: "c5e1107a1b08424596d370aefbbc27ee" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34469 } }
I20260812 17:50:53.984776 2239 raft_consensus.cc:374] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:50:53.984853 2239 raft_consensus.cc:729] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e1107a1b08424596d370aefbbc27ee, State: Initialized, Role: FOLLOWER
I20260812 17:50:53.985553 2239 consensus_queue.cc:260] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0130461f10f94f43a966b8bc44d5da95" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33089 } } peers { permanent_uuid: "026617057df24ac384cf7e9e23165ecf" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45095 } } peers { permanent_uuid: "c5e1107a1b08424596d370aefbbc27ee" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34469 } }
I20260812 17:50:53.986088 2238 raft_consensus.cc:348] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0130461f10f94f43a966b8bc44d5da95" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33089 } } peers { permanent_uuid: "026617057df24ac384cf7e9e23165ecf" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45095 } } peers { permanent_uuid: "c5e1107a1b08424596d370aefbbc27ee" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34469 } }
I20260812 17:50:53.986438 2238 raft_consensus.cc:374] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:50:53.986548 2238 raft_consensus.cc:729] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0130461f10f94f43a966b8bc44d5da95, State: Initialized, Role: FOLLOWER
I20260812 17:50:53.986878 2239 ts_tablet_manager.cc:1428] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260812 17:50:53.987171 2238 consensus_queue.cc:260] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0130461f10f94f43a966b8bc44d5da95" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33089 } } peers { permanent_uuid: "026617057df24ac384cf7e9e23165ecf" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45095 } } peers { permanent_uuid: "c5e1107a1b08424596d370aefbbc27ee" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34469 } }
I20260812 17:50:53.987313 1985 heartbeater.cc:503] Master 127.13.38.62:35119 was elected leader, sending a full tablet report...
I20260812 17:50:53.988512 2238 ts_tablet_manager.cc:1428] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260812 17:50:53.989081 2194 heartbeater.cc:503] Master 127.13.38.62:35119 was elected leader, sending a full tablet report...
I20260812 17:50:53.995839 2246 raft_consensus.cc:482] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:50:53.996104 2246 raft_consensus.cc:504] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0130461f10f94f43a966b8bc44d5da95" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33089 } } peers { permanent_uuid: "026617057df24ac384cf7e9e23165ecf" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45095 } } peers { permanent_uuid: "c5e1107a1b08424596d370aefbbc27ee" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34469 } }
I20260812 17:50:53.999717 2246 leader_election.cc:288] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0130461f10f94f43a966b8bc44d5da95 (127.13.38.3:33089), 026617057df24ac384cf7e9e23165ecf (127.13.38.1:45095)
W20260812 17:50:54.003700 1792 tablet.cc:2369] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:50:54.004977 2232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:53.999494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:32871 (local address 127.13.38.1:45095)
0812 17:50:53.999678 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:53.999683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:54.000068 (+ 385us) server_negotiation.cc:407] Connection header received
0812 17:50:54.000284 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:54.000289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:54.000348 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:54.000417 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:50:54.000997 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:54.001786 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:54.003556 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:54.004141 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:54.004240 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:54.004316 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:54.004742 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 17:50:54.004838 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 17:50:54.005470 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:53.998951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45095 (local address 127.13.38.2:32871)
0812 17:50:53.999980 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:53.999995 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:54.000152 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:54.000577 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:54.000580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:54.000595 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:50:54.000829 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:54.000838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:54.001952 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:54.001955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:54.003374 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:54.003388 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:54.004471 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:54.004475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:54.004583 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:54.004610 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 17:50:54.005209 (+ 599us) client_negotiation.cc:240] Negotiation successful
0812 17:50:54.005305 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"thread_start_us":185,"threads_started":1}
I20260812 17:50:54.006234 1704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b77df7e76e486caa3a117e0c13441c" candidate_uuid: "c5e1107a1b08424596d370aefbbc27ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026617057df24ac384cf7e9e23165ecf" is_pre_election: true
I20260812 17:50:54.006654 1704 raft_consensus.cc:2454] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5e1107a1b08424596d370aefbbc27ee in term 0.
I20260812 17:50:54.007283 1820 leader_election.cc:302] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026617057df24ac384cf7e9e23165ecf, c5e1107a1b08424596d370aefbbc27ee; no voters:
I20260812 17:50:54.007714 2246 raft_consensus.cc:2781] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:50:54.007812 2246 raft_consensus.cc:482] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:50:54.007872 2246 raft_consensus.cc:3037] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:50:54.009424 2246 raft_consensus.cc:504] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0130461f10f94f43a966b8bc44d5da95" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33089 } } peers { permanent_uuid: "026617057df24ac384cf7e9e23165ecf" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45095 } } peers { permanent_uuid: "c5e1107a1b08424596d370aefbbc27ee" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34469 } }
I20260812 17:50:54.009845 2246 leader_election.cc:288] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [CANDIDATE]: Term 1 election: Requested vote from peers 0130461f10f94f43a966b8bc44d5da95 (127.13.38.3:33089), 026617057df24ac384cf7e9e23165ecf (127.13.38.1:45095)
I20260812 17:50:54.010366 1704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b77df7e76e486caa3a117e0c13441c" candidate_uuid: "c5e1107a1b08424596d370aefbbc27ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026617057df24ac384cf7e9e23165ecf"
I20260812 17:50:54.010519 1704 raft_consensus.cc:3037] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:50:54.011683 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:53.998435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:37333 (local address 127.13.38.3:33089)
0812 17:50:53.998575 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:53.998579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:53.999616 (+ 1037us) server_negotiation.cc:407] Connection header received
0812 17:50:54.000470 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:54.000473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:54.000535 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:54.000585 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:50:54.008126 (+ 7541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:54.008860 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:54.010296 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:54.010918 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:54.011008 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:54.011080 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:54.011431 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 17:50:54.011544 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 17:50:54.012082 1704 raft_consensus.cc:2454] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5e1107a1b08424596d370aefbbc27ee in term 1.
I20260812 17:50:54.012177 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:53.997989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:33089 (local address 127.13.38.2:37333)
0812 17:50:53.999490 (+ 1501us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:53.999507 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:54.000331 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:54.007255 (+ 6924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:54.007258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:54.007289 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:50:54.007423 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:54.007431 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:54.009037 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:54.009041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:54.010149 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:54.010159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:54.011182 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:54.011186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:54.011283 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:54.011307 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 17:50:54.011891 (+ 584us) client_negotiation.cc:240] Negotiation successful
0812 17:50:54.012004 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1412,"thread_start_us":124,"threads_started":1}
I20260812 17:50:54.012466 1820 leader_election.cc:302] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026617057df24ac384cf7e9e23165ecf, c5e1107a1b08424596d370aefbbc27ee; no voters:
I20260812 17:50:54.012665 2246 raft_consensus.cc:2781] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:50:54.012952 2246 raft_consensus.cc:686] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [term 1 LEADER]: Becoming Leader. State: Replica: c5e1107a1b08424596d370aefbbc27ee, State: Running, Role: LEADER
I20260812 17:50:54.013132 2117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b77df7e76e486caa3a117e0c13441c" candidate_uuid: "c5e1107a1b08424596d370aefbbc27ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0130461f10f94f43a966b8bc44d5da95" is_pre_election: true
I20260812 17:50:54.013389 2246 consensus_queue.cc:237] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0130461f10f94f43a966b8bc44d5da95" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33089 } } peers { permanent_uuid: "026617057df24ac384cf7e9e23165ecf" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45095 } } peers { permanent_uuid: "c5e1107a1b08424596d370aefbbc27ee" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34469 } }
I20260812 17:50:54.013484 2117 raft_consensus.cc:2454] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5e1107a1b08424596d370aefbbc27ee in term 0.
I20260812 17:50:54.013833 2105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b77df7e76e486caa3a117e0c13441c" candidate_uuid: "c5e1107a1b08424596d370aefbbc27ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0130461f10f94f43a966b8bc44d5da95"
I20260812 17:50:54.013972 2105 raft_consensus.cc:3037] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:50:54.015342 2105 raft_consensus.cc:2454] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5e1107a1b08424596d370aefbbc27ee in term 1.
I20260812 17:50:54.017217 1505 catalog_manager.cc:5495] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee reported cstate change: term changed from 0 to 1, leader changed from <none> to c5e1107a1b08424596d370aefbbc27ee (127.13.38.2). New cstate: current_term: 1 leader_uuid: "c5e1107a1b08424596d370aefbbc27ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0130461f10f94f43a966b8bc44d5da95" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "026617057df24ac384cf7e9e23165ecf" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5e1107a1b08424596d370aefbbc27ee" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34469 } health_report { overall_health: HEALTHY } } }
Aug 12 17:50:54 dist-test-slave-sb2z krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557053, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for HTTP/127.13.38.20@KRBTEST.COM
W20260812 17:50:54.044478 1987 tablet.cc:2369] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:50:54.061951 2235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:54.055802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48306 (local address 127.13.38.2:34469)
0812 17:50:54.055961 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:54.055972 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:54.055988 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:50:54.056232 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:54.056238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:54.056353 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:54.056536 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:50:54.056927 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:54.057723 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:54.059013 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:54.059402 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:54.059579 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:54.060058 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:50:54.060763 (+ 705us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:50:54.060915 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:54.061683 (+ 768us) server_negotiation.cc:299] Negotiation successful
0812 17:50:54.061760 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 17:50:54.079393 1704 raft_consensus.cc:1261] T 25b77df7e76e486caa3a117e0c13441c P 026617057df24ac384cf7e9e23165ecf [term 1 FOLLOWER]: Refusing update from remote peer c5e1107a1b08424596d370aefbbc27ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:50:54.080699 2105 raft_consensus.cc:1261] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95 [term 1 FOLLOWER]: Refusing update from remote peer c5e1107a1b08424596d370aefbbc27ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:50:54.081956 2258 consensus_queue.cc:1035] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "0130461f10f94f43a966b8bc44d5da95" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 33089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:50:54.088337 2258 consensus_queue.cc:1035] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "026617057df24ac384cf7e9e23165ecf" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 17:50:54.097364 2196 tablet.cc:2369] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:50:54.110567 2265 mvcc.cc:205] Tried to move back new op lower bound from 7317737693499469824 to 7317737693242728448. Current Snapshot: MvccSnapshot[applied={T|T < 7317737693499469824}]
I20260812 17:50:54.120683 1505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:50:54.120921 1505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:50:54.123637 2262 mvcc.cc:205] Tried to move back new op lower bound from 7317737693499469824 to 7317737693242728448. Current Snapshot: MvccSnapshot[applied={T|T < 7317737693499469824}]
I20260812 17:50:54.124462 1505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3449a937f5c4ed9b0a7e861e69bf0ad: Sending DeleteTablet for 3 replicas of tablet 25b77df7e76e486caa3a117e0c13441c
I20260812 17:50:54.125415 2086 tablet_service.cc:1514] Processing DeleteTablet for tablet 25b77df7e76e486caa3a117e0c13441c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:50:54 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:55878
I20260812 17:50:54.125905 1698 tablet_service.cc:1514] Processing DeleteTablet for tablet 25b77df7e76e486caa3a117e0c13441c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:50:54 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:50104
I20260812 17:50:54.126461 1889 tablet_service.cc:1514] Processing DeleteTablet for tablet 25b77df7e76e486caa3a117e0c13441c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:50:54 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:48304
I20260812 17:50:54.128276 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 1560
I20260812 17:50:54.135820 2276 tablet_replica.cc:330] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95: stopping tablet replica
I20260812 17:50:54.136323 2276 raft_consensus.cc:2229] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:50:54.136683 2276 raft_consensus.cc:2258] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 17:50:54.153707 1482 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:45095 (error 108)
I20260812 17:50:54.155767 2278 tablet_replica.cc:330] T 25b77df7e76e486caa3a117e0c13441c P c5e1107a1b08424596d370aefbbc27ee: stopping tablet replica
I20260812 17:50:54.156097 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 1793
I20260812 17:50:54.163305 2234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:54.162913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45095 (local address 127.0.0.1:50118)
0812 17:50:54.163133 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:54.163208 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:45095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260812 17:50:54.163676 1482 catalog_manager.cc:4584] TS 026617057df24ac384cf7e9e23165ecf (127.13.38.1:45095): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 25b77df7e76e486caa3a117e0c13441c: Network error: Client connection negotiation failed: client connection to 127.13.38.1:45095: connect: Connection refused (error 111)
I20260812 17:50:54.174115 2276 ts_tablet_manager.cc:1905] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:50:54.177433 2276 ts_tablet_manager.cc:1918] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:50:54.177471 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 1992
I20260812 17:50:54.177567 2276 log.cc:1196] T 25b77df7e76e486caa3a117e0c13441c P 0130461f10f94f43a966b8bc44d5da95: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/25b77df7e76e486caa3a117e0c13441c
I20260812 17:50:54.177735 2234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:54.177365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:34469 (local address 127.0.0.1:48320)
0812 17:50:54.177592 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:54.177651 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.2:34469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260812 17:50:54.208778 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 1452
I20260812 17:50:54.224722 13464 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 17:50:54.571090 13464 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 17:50:54.571269 13464 mini_ranger.cc:66] Stopping Ranger...
I20260812 17:50:54.927201 13464 mini_ranger.cc:68] Stopped Ranger
2026-08-12 17:50:54.927 UTC [32482] LOG: received smart shutdown request
2026-08-12 17:50:54.932 UTC [32482] LOG: background worker "logical replication launcher" (PID 32496) exited with exit code 1
2026-08-12 17:50:54.933 UTC [32488] LOG: shutting down
2026-08-12 17:50:55.126 UTC [32482] LOG: database system is shut down
2026-08-12T17:50:55Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53412 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:50:55 dist-test-slave-sb2z krb5kdc[2293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:50:55 dist-test-slave-sb2z krb5kdc[2293](info): set up 2 sockets
Aug 12 17:50:55 dist-test-slave-sb2z krb5kdc[2293](info): commencing operation
krb5kdc: starting...
W20260812 17:50:56.554710 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:50:56 dist-test-slave-sb2z krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:50:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:50:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:50:56.695426 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:44389
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:44389,127.13.38.61:36593,127.13.38.60:38153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:50:56.821136 2309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:50:56.821467 2309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:50:56.821516 2309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:50:56.825786 2309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:50:56.825870 2309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:50:56.825891 2309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:50:56.825908 2309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:50:56.825925 2309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:50:56.831707 2309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:44389,127.13.38.61:36593,127.13.38.60:38153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:44389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:50:56.832993 2309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:50:56.834173 2309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:50:56.843842 2318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:56.843946 2309 server_base.cc:1034] running on GCE node
W20260812 17:50:56.843832 2319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:56.843832 2321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:56.844868 2309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:50:56.846233 2309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:50:56.847424 2309 hybrid_clock.cc:648] HybridClock initialized: now 1786557056847392 us; error 45 us; skew 500 ppm
Aug 12 17:50:56 dist-test-slave-sb2z krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557056, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:56.861330 2309 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:50:56.862507 2309 webserver.cc:460] Webserver started at http://127.13.38.62:37845/ using document root <none> and password file <none>
I20260812 17:50:56.863168 2309 fs_manager.cc:359] Metadata directory not provided
I20260812 17:50:56.863276 2309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:50:56.863611 2309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:50:56.865916 2309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "cbfb47b2ab1148098378630270c783d8"
format_stamp: "Formatted at 2026-08-12 17:50:56 on dist-test-slave-sb2z"
server_key: "c7fcd81643c4c476d998e4fd4e0ae64a"
server_key_iv: "66af289f1d4a14c3ee13a617e0398781"
server_key_version: "encryptionkey@0"
I20260812 17:50:56.866568 2309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "cbfb47b2ab1148098378630270c783d8"
format_stamp: "Formatted at 2026-08-12 17:50:56 on dist-test-slave-sb2z"
server_key: "c7fcd81643c4c476d998e4fd4e0ae64a"
server_key_iv: "66af289f1d4a14c3ee13a617e0398781"
server_key_version: "encryptionkey@0"
I20260812 17:50:56.870780 2309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 17:50:56.874006 2330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:56.875583 2309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:50:56.875749 2309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "cbfb47b2ab1148098378630270c783d8"
format_stamp: "Formatted at 2026-08-12 17:50:56 on dist-test-slave-sb2z"
server_key: "c7fcd81643c4c476d998e4fd4e0ae64a"
server_key_iv: "66af289f1d4a14c3ee13a617e0398781"
server_key_version: "encryptionkey@0"
I20260812 17:50:56.875880 2309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:50:56.900899 2309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:50:56.904726 2309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:50:56.904950 2309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:50:56.912943 2309 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:44389
I20260812 17:50:56.912971 2416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:44389 every 8 connection(s)
I20260812 17:50:56.913981 2309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:50:56.915019 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 2309
I20260812 17:50:56.915161 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:50:56.915448 13464 external_mini_cluster.cc:1428] Setting key edd6f23c69eeee5cf3b2ced76420cc60
I20260812 17:50:56.917286 2417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:50:56.923489 2417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.38.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.61@KRBTEST.COM" created.
Aug 12 17:50:56 dist-test-slave-sb2z krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557056, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:50:56.942401 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:56.926757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44389 (local address 127.0.0.1:51068)
0812 17:50:56.927460 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:56.927512 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:56.928514 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:56.929424 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:56.929475 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:56.929974 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:56.930402 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:56.930456 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:56.931722 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:56.931730 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:56.932736 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:56.932748 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:56.933529 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:56.933534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:56.933671 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:56.934371 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:56.934390 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:56.936465 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:56.939099 (+ 2634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:56.939108 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:56.939118 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:56.939369 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:56.939828 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:56.939830 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:56.939832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:56.939960 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:56.940685 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:56.940692 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:56.940907 (+ 215us) client_negotiation.cc:769] Sending connection context
0812 17:50:56.941114 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 17:50:56.941532 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":97,"spinlock_wait_cycles":62208,"thread_start_us":184,"threads_started":1}
I20260812 17:50:56.942402 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:56.926757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51068 (local address 127.13.38.62:44389)
0812 17:50:56.927386 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:56.927406 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:56.927679 (+ 273us) server_negotiation.cc:407] Connection header received
0812 17:50:56.928890 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:56.928894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:56.929130 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:56.929372 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:56.930636 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:56.931519 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:56.932888 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:56.933317 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:56.933543 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:56.936662 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:56.936672 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:56.936694 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:56.936730 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:56.938957 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:56.939507 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:56.939514 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:56.939516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:56.939700 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:56.940076 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:56.940082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:56.940084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:56.940540 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:56.940659 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:56.941214 (+ 555us) server_negotiation.cc:299] Negotiation successful
0812 17:50:56.941409 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":182,"threads_started":1}
I20260812 17:50:56.945201 2417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:50:56.946130 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:56.945792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:36593 (local address 127.0.0.1:39496)
0812 17:50:56.945969 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:56.946055 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 17:50:56.946393 2334 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.38.61:36593: connect: Connection refused (error 111)
I20260812 17:50:56.947646 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:56.947312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:36593 (local address 127.0.0.1:39504)
0812 17:50:56.947459 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:56.947510 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 17:50:56.947803 2417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.61:36593: Network error: Client connection negotiation failed: client connection to 127.13.38.61:36593: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.61.keytab.
Entry for principal kudu/127.13.38.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.61.keytab.
I20260812 17:50:56.951613 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.61:36593
--webserver_interface=127.13.38.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.61.keytab
--principal=kudu/127.13.38.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:44389,127.13.38.61:36593,127.13.38.60:38153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:50:56.997133 2417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } attempt: 1
I20260812 17:50:56.998219 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:56.997833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:36593 (local address 127.0.0.1:39506)
0812 17:50:56.998032 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:56.998112 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 17:50:56.999079 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:56.998831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:36593 (local address 127.0.0.1:39516)
0812 17:50:56.998943 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:56.998998 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 17:50:56.999251 2417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.61:36593: Network error: Client connection negotiation failed: client connection to 127.13.38.61:36593: connect: Connection refused (error 111)
I20260812 17:50:57.067492 2417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } attempt: 2
I20260812 17:50:57.068537 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.068168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:36593 (local address 127.0.0.1:39520)
0812 17:50:57.068368 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.068443 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 17:50:57.069355 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.069108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:36593 (local address 127.0.0.1:39536)
0812 17:50:57.069224 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.069278 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 17:50:57.069545 2417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.61:36593: Network error: Client connection negotiation failed: client connection to 127.13.38.61:36593: connect: Connection refused (error 111)
W20260812 17:50:57.073590 2427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:50:57.073896 2427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:50:57.073982 2427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:50:57.078150 2427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:50:57.078248 2427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:50:57.078285 2427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:50:57.078343 2427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:50:57.078382 2427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:50:57.084131 2427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:44389,127.13.38.61:36593,127.13.38.60:38153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.61.keytab
--principal=kudu/127.13.38.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.61:36593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.38.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:50:57.085317 2427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:50:57.086386 2427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 17:50:57.095644 2427 server_base.cc:1034] running on GCE node
W20260812 17:50:57.095566 2439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:57.095566 2435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:57.095558 2432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:57.096654 2427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:50:57.097812 2427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:50:57.099006 2427 hybrid_clock.cc:648] HybridClock initialized: now 1786557057098970 us; error 51 us; skew 500 ppm
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557057, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:57.102234 2427 init.cc:377] Logged in from keytab as kudu/127.13.38.61@KRBTEST.COM (short username kudu)
I20260812 17:50:57.103222 2427 webserver.cc:460] Webserver started at http://127.13.38.61:36745/ using document root <none> and password file <none>
I20260812 17:50:57.103864 2427 fs_manager.cc:359] Metadata directory not provided
I20260812 17:50:57.103943 2427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:50:57.104126 2427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:50:57.106254 2427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/data/instance:
uuid: "53f625f9e53d45dbb9a43069bb539d1c"
format_stamp: "Formatted at 2026-08-12 17:50:57 on dist-test-slave-sb2z"
server_key: "125af677185c5118e638ce70f22db235"
server_key_iv: "06698391b502230e2c2e8d8e58b2f5dc"
server_key_version: "encryptionkey@0"
I20260812 17:50:57.106794 2427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/wal/instance:
uuid: "53f625f9e53d45dbb9a43069bb539d1c"
format_stamp: "Formatted at 2026-08-12 17:50:57 on dist-test-slave-sb2z"
server_key: "125af677185c5118e638ce70f22db235"
server_key_iv: "06698391b502230e2c2e8d8e58b2f5dc"
server_key_version: "encryptionkey@0"
I20260812 17:50:57.110615 2427 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:50:57.113245 2455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:57.114617 2427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 17:50:57.114858 2427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/wal
uuid: "53f625f9e53d45dbb9a43069bb539d1c"
format_stamp: "Formatted at 2026-08-12 17:50:57 on dist-test-slave-sb2z"
server_key: "125af677185c5118e638ce70f22db235"
server_key_iv: "06698391b502230e2c2e8d8e58b2f5dc"
server_key_version: "encryptionkey@0"
I20260812 17:50:57.114979 2427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:50:57.131062 2427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:50:57.134509 2427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:50:57.134743 2427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:50:57.142455 2427 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.61:36593
I20260812 17:50:57.142477 2531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.61:36593 every 8 connection(s)
I20260812 17:50:57.143469 2427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
I20260812 17:50:57.146395 2533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:50:57.150098 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 2427
I20260812 17:50:57.150283 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-1/wal/instance
I20260812 17:50:57.150568 13464 external_mini_cluster.cc:1428] Setting key 3870dc5d32767b32cc12e45ad807981f
I20260812 17:50:57.150395 2533 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:50:57.160727 2417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } attempt: 3
WARNING: no policy specified for kudu/127.13.38.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.60@KRBTEST.COM" created.
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557057, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.61@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557056, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for kudu/127.13.38.61@KRBTEST.COM
I20260812 17:50:57.169749 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.152472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51078 (local address 127.13.38.62:44389)
0812 17:50:57.152669 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.152678 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.153673 (+ 995us) server_negotiation.cc:407] Connection header received
0812 17:50:57.154960 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.154963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.155016 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.155102 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.156935 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.157461 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.159321 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.159617 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.159701 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.165544 (+ 5843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.165550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.165554 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.165584 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.167396 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.168059 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.168065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.168069 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.168152 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.168524 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.168528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.168529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.168722 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.168834 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.169419 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.169566 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 17:50:57.170871 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.152935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44389 (local address 127.0.0.1:51078)
0812 17:50:57.153502 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.153549 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:57.154691 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:57.155316 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:57.155327 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:57.155900 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:57.156738 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.156762 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.157614 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.157619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.159170 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.159186 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.159784 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.159790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.160021 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.160681 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:57.160709 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:57.165369 (+ 4660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:57.167564 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.167575 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.167593 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.167966 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.168253 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.168257 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.168259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.168413 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.168837 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:57.168846 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:57.169133 (+ 287us) client_negotiation.cc:769] Sending connection context
0812 17:50:57.169334 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 17:50:57.169696 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":144,"threads_started":1}
I20260812 17:50:57.172360 2533 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:50:57.172518 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.161248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:36593 (local address 127.0.0.1:39550)
0812 17:50:57.161441 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.161457 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:57.161553 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:57.162310 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:57.162312 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:57.162459 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:57.162568 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.162573 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.163638 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.163640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.164142 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.164150 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.165011 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.165015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.165136 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.165695 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:57.165718 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:57.167870 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:57.170373 (+ 2503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.170376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.170378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.170629 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.171030 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.171034 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.171037 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.171141 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.171853 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:57.171859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:57.171995 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 17:50:57.172121 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 17:50:57.172293 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260812 17:50:57.173888 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.161347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39550 (local address 127.13.38.61:36593)
0812 17:50:57.161931 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.161939 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.161961 (+ 22us) server_negotiation.cc:407] Connection header received
0812 17:50:57.161997 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.162001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.162161 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.162296 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.162697 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.163443 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.164261 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.164886 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.165007 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.168045 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.168057 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.168073 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.168104 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.170248 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.170765 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.170774 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.170779 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.170881 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.171263 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.171268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.171273 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.171651 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.171795 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.173490 (+ 1695us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.173657 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":169,"threads_started":1}
I20260812 17:50:57.176173 2417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:50:57.176890 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.176649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36496)
0812 17:50:57.176769 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.176822 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 17:50:57.177699 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.177457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36500)
0812 17:50:57.177598 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.177651 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260812 17:50:57.177838 2417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:38153: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557057, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.61@KRBTEST.COM for kudu/127.13.38.61@KRBTEST.COM
I20260812 17:50:57.183607 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.173003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:36593 (local address 127.0.0.1:39558)
0812 17:50:57.173146 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.173157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:57.173222 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:57.173681 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:57.173684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:57.173856 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:57.173956 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.173962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.174941 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.174947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.175468 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.175476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.176052 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.176055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.176154 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.176698 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:57.176711 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:57.179233 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:57.181856 (+ 2623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.181858 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.181860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.182085 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.182491 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.182494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.182496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.182593 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.183025 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:57.183031 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:57.183166 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 17:50:57.183270 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 17:50:57.183385 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 17:50:57.183643 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.173177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39558 (local address 127.13.38.61:36593)
0812 17:50:57.173488 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.173495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.173505 (+ 10us) server_negotiation.cc:407] Connection header received
0812 17:50:57.173531 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.173533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.173568 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.173673 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.174094 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.174851 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.175655 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.175936 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.176392 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.179396 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.179399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.179401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.179422 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.181742 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.182244 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.182247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.182250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.182357 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.182687 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.182690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.182691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.182883 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.182964 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.183334 (+ 370us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.183439 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":125,"threads_started":1}
I20260812 17:50:57.184448 2533 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:50:57.185190 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.184905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36516)
0812 17:50:57.185027 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.185126 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 17:50:57.185441 2456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.60.keytab.
Entry for principal kudu/127.13.38.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.60.keytab.
I20260812 17:50:57.186369 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.186155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36518)
0812 17:50:57.186268 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.186319 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 17:50:57.186538 2533 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:38153: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
I20260812 17:50:57.187207 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.60:38153
--webserver_interface=127.13.38.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.60.keytab
--principal=kudu/127.13.38.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:42451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:44389,127.13.38.61:36593,127.13.38.60:38153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:50:57.209026 2417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } attempt: 1
I20260812 17:50:57.209975 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.209645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36522)
0812 17:50:57.209805 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.209881 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 17:50:57.210795 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.210578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36536)
0812 17:50:57.210691 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.210738 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 17:50:57.210980 2417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:38153: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
I20260812 17:50:57.235832 2533 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } attempt: 1
I20260812 17:50:57.236923 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.236566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36546)
0812 17:50:57.236759 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.236831 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 17:50:57.237759 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.237533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36550)
0812 17:50:57.237650 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.237702 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 17:50:57.237960 2533 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:38153: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
I20260812 17:50:57.286175 2417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } attempt: 2
I20260812 17:50:57.287249 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.286817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36554)
0812 17:50:57.287078 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.287165 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260812 17:50:57.288086 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.287849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36562)
0812 17:50:57.287973 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.288029 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 17:50:57.288268 2417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:38153: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
I20260812 17:50:57.306162 2533 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } attempt: 2
I20260812 17:50:57.307176 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.306828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36572)
0812 17:50:57.307014 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.307094 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 17:50:57.307998 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.307781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36574)
0812 17:50:57.307893 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.307946 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 17:50:57.308120 2533 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:38153: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
W20260812 17:50:57.309823 2544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:50:57.310135 2544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:50:57.310240 2544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:50:57.314436 2544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:50:57.314515 2544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:50:57.314551 2544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:50:57.314605 2544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:50:57.314654 2544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:50:57.321925 2544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:42451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:44389,127.13.38.61:36593,127.13.38.60:38153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.60.keytab
--principal=kudu/127.13.38.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.60:38153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.38.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:50:57.323474 2544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:50:57.324843 2544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:50:57.335685 2557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:50:57.335625 2553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:57.336007 2544 server_base.cc:1034] running on GCE node
W20260812 17:50:57.335520 2551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:50:57.337083 2544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:50:57.338296 2544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:50:57.339576 2544 hybrid_clock.cc:648] HybridClock initialized: now 1786557057339509 us; error 35 us; skew 500 ppm
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557057, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:50:57.342618 2544 init.cc:377] Logged in from keytab as kudu/127.13.38.60@KRBTEST.COM (short username kudu)
I20260812 17:50:57.343647 2544 webserver.cc:460] Webserver started at http://127.13.38.60:43215/ using document root <none> and password file <none>
I20260812 17:50:57.344254 2544 fs_manager.cc:359] Metadata directory not provided
I20260812 17:50:57.344355 2544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:50:57.344614 2544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:50:57.346782 2544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/data/instance:
uuid: "035082482d4b4b3b93bea3252da512ea"
format_stamp: "Formatted at 2026-08-12 17:50:57 on dist-test-slave-sb2z"
server_key: "dbbdad8273a22138d9aa3c9408a47cb3"
server_key_iv: "4e4f2ea0fd3889e0c4beee60417f68cf"
server_key_version: "encryptionkey@0"
I20260812 17:50:57.347386 2544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/wal/instance:
uuid: "035082482d4b4b3b93bea3252da512ea"
format_stamp: "Formatted at 2026-08-12 17:50:57 on dist-test-slave-sb2z"
server_key: "dbbdad8273a22138d9aa3c9408a47cb3"
server_key_iv: "4e4f2ea0fd3889e0c4beee60417f68cf"
server_key_version: "encryptionkey@0"
I20260812 17:50:57.351413 2544 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 17:50:57.354054 2566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:50:57.355291 2544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 17:50:57.355443 2544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/wal
uuid: "035082482d4b4b3b93bea3252da512ea"
format_stamp: "Formatted at 2026-08-12 17:50:57 on dist-test-slave-sb2z"
server_key: "dbbdad8273a22138d9aa3c9408a47cb3"
server_key_iv: "4e4f2ea0fd3889e0c4beee60417f68cf"
server_key_version: "encryptionkey@0"
I20260812 17:50:57.355630 2544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:50:57.386214 2544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:50:57.387449 2417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } attempt: 3
I20260812 17:50:57.388320 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.388003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36582)
0812 17:50:57.388179 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.388243 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260812 17:50:57.389184 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.388944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36596)
0812 17:50:57.389060 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.389114 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 17:50:57.389362 2417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:38153: Network error: Client connection negotiation failed: client connection to 127.13.38.60:38153: connect: Connection refused (error 111)
I20260812 17:50:57.390244 2544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:50:57.390462 2544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:50:57.397980 2544 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.60:38153
I20260812 17:50:57.398007 2648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.60:38153 every 8 connection(s)
I20260812 17:50:57.399014 2544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
I20260812 17:50:57.399302 2533 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } attempt: 3
I20260812 17:50:57.402459 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:50:57.406543 2652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557057, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.61@KRBTEST.COM for kudu/127.13.38.60@KRBTEST.COM
I20260812 17:50:57.407702 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 2544
I20260812 17:50:57.407850 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786556880371321-13464-0/minicluster-data/master-2/wal/instance
I20260812 17:50:57.408169 13464 external_mini_cluster.cc:1428] Setting key f19787a859880b12f38016be228e5699
I20260812 17:50:57.412801 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.399913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36610)
0812 17:50:57.400079 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.400099 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:57.400205 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:57.402640 (+ 2435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:57.402644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:57.402865 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:57.402983 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.402991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.404424 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.404430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.404999 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.405011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.405817 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.405824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.405907 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.406492 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:57.406505 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:57.408028 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:57.410682 (+ 2654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.410685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.410688 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.410932 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.411304 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.411308 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.411310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.411408 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.412211 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:57.412214 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:57.412321 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 17:50:57.412434 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 17:50:57.412590 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 17:50:57.414160 2654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.400714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36610 (local address 127.13.38.60:38153)
0812 17:50:57.401214 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.401231 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.401272 (+ 41us) server_negotiation.cc:407] Connection header received
0812 17:50:57.401939 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.401955 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.402301 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.402719 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.403183 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.404280 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.405147 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.405674 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.405860 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.408261 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.408270 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.408287 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.408320 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.410557 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.411036 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.411042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.411048 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.411194 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.411518 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.411521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.411554 (+ 33us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.412087 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.412222 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.412609 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.413065 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":171,"threads_started":1}
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.60@KRBTEST.COM
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557057, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.60@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:50:57.419173 2533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c: Bootstrap starting.
I20260812 17:50:57.420933 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.408916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51080 (local address 127.13.38.62:44389)
0812 17:50:57.409079 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.409086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.409705 (+ 619us) server_negotiation.cc:407] Connection header received
0812 17:50:57.409925 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.409928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.409983 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.410062 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.411022 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.411741 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.412930 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.413248 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.413325 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.417250 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.417254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.417256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.417284 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.418965 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.419569 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.419575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.419577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.419674 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.419975 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.419978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.419979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.420191 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.420271 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.420693 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.420802 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 17:50:57.421021 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.408570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44389 (local address 127.0.0.1:51080)
0812 17:50:57.409568 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.409609 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:57.409821 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:57.410176 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:57.410182 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:57.410734 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:57.410893 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.410903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.411874 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.411877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.412782 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.412791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.413484 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.413487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.413592 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.414181 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:57.414198 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:57.417071 (+ 2873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:57.419095 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.419105 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.419111 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.419421 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.419772 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.419775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.419777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.419879 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.420302 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:57.420308 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:57.420441 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 17:50:57.420614 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 17:50:57.420827 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"thread_start_us":174,"threads_started":1}
I20260812 17:50:57.422017 2533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c: Neither blocks nor log segments found. Creating new log.
I20260812 17:50:57.422205 2652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:50:57.422996 2533 log.cc:824] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c: Log is configured to *not* fsync() on all Append() calls
I20260812 17:50:57.425347 2533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c: No bootstrap required, opened a new log
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557057, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.60@KRBTEST.COM for kudu/127.13.38.61@KRBTEST.COM
I20260812 17:50:57.430001 2533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } }
I20260812 17:50:57.430318 2533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:50:57.430437 2533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f625f9e53d45dbb9a43069bb539d1c, State: Initialized, Role: FOLLOWER
I20260812 17:50:57.431167 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.409092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36620 (local address 127.13.38.60:38153)
0812 17:50:57.409387 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.409397 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.409691 (+ 294us) server_negotiation.cc:407] Connection header received
0812 17:50:57.409837 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.409842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.409892 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.410012 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.410948 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.411700 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.412753 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.413130 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.413227 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.423561 (+ 10334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.423566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.423568 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.423592 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.428947 (+ 5355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.429583 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.429586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.429588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.429678 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.430030 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.430032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.430035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.430289 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.430538 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.430795 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.430929 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":95,"threads_started":1}
I20260812 17:50:57.431133 2533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } }
I20260812 17:50:57.431746 2660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } } }
I20260812 17:50:57.432006 2660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:50:57.432510 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.422826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39562 (local address 127.13.38.61:36593)
0812 17:50:57.422953 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.422961 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.422979 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:50:57.423143 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.423146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.423195 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.423278 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.423828 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.424394 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.425414 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.425664 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.425751 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.428134 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.428140 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.428146 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.428176 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.430452 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.430969 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.430973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.430974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.431068 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.431395 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.431399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.431401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.431713 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.431793 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.432181 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.432339 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 17:50:57.432864 2533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:50:57.432986 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.422750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:36593 (local address 127.0.0.1:39562)
0812 17:50:57.422926 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.422940 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:57.423033 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:57.423319 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:57.423325 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:57.423519 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:57.423689 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.423697 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.424513 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.424517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.425282 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.425294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.425795 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.425799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.425902 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.426387 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:57.426401 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:57.427981 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:57.430596 (+ 2615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.430600 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.430602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.430848 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.431193 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.431196 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.431199 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.431290 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.431875 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:57.431880 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:57.431983 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 17:50:57.432654 (+ 671us) client_negotiation.cc:240] Negotiation successful
0812 17:50:57.432808 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 17:50:57.434270 2652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
W20260812 17:50:57.439205 2676 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:50:57.439319 2676 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:50:57.437601 2427 master_runner.cc:186] Error getting master registration for 127.13.38.62:44389: OK, instance_id { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" instance_seqno: 1786557056910710 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557057, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.60@KRBTEST.COM for kudu/127.13.38.60@KRBTEST.COM
I20260812 17:50:57.441010 2427 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 17:50:57.443658 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.433048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51094 (local address 127.13.38.62:44389)
0812 17:50:57.433172 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.433182 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.433193 (+ 11us) server_negotiation.cc:407] Connection header received
0812 17:50:57.433230 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.433232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.433270 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.433368 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.434228 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.434909 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.436078 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.436408 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.436509 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.439705 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.439711 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.439716 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.439751 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.441710 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.442231 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.442235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.442237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.442378 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.442731 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.442735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.442737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.442937 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.443166 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.443314 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.443440 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 17:50:57.445238 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.434872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36626 (local address 127.13.38.60:38153)
0812 17:50:57.434999 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.435005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.435049 (+ 44us) server_negotiation.cc:407] Connection header received
0812 17:50:57.435193 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.435196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.435240 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.435312 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.435805 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.436527 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.438011 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.438385 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.438468 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.441583 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.441586 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.441589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.441615 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.443318 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.443849 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.443852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.443855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.443940 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.444250 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.444256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.444258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.444519 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.444633 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.444939 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.445081 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 17:50:57.445770 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.434816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36626)
0812 17:50:57.434990 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.435004 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:57.435099 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:57.435403 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:57.435406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:57.435615 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:57.435697 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.435703 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.436642 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.436644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.437847 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.437862 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.438606 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.438610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.438698 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.439202 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:57.439221 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:57.441408 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:57.443444 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.443447 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.443449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.443741 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.444045 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.444048 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.444050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.444129 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.444621 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:57.444624 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:57.444719 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 17:50:57.445531 (+ 812us) client_negotiation.cc:240] Negotiation successful
0812 17:50:57.445668 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260812 17:50:57.449132 2652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea: Bootstrap starting.
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.61@KRBTEST.COM
I20260812 17:50:57.452396 2652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea: Neither blocks nor log segments found. Creating new log.
I20260812 17:50:57.453258 2652 log.cc:824] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea: Log is configured to *not* fsync() on all Append() calls
I20260812 17:50:57.455431 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.444989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.13.38.61:36593)
0812 17:50:57.445110 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.445115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.445129 (+ 14us) server_negotiation.cc:407] Connection header received
0812 17:50:57.445169 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.445172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.445216 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.445306 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.446101 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.446915 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.448018 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.448365 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.448487 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.451437 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.451441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.451444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.451478 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.453377 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.453871 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.453875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.453877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.454013 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.454447 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.454451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.454452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.454657 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.454885 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.455106 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.455234 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 17:50:57.455752 2652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea: No bootstrap required, opened a new log
I20260812 17:50:57.458876 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:50:57.460104 2652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } }
I20260812 17:50:57.460397 2652 raft_consensus.cc:374] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:50:57.460455 2652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 035082482d4b4b3b93bea3252da512ea, State: Initialized, Role: FOLLOWER
I20260812 17:50:57.461014 2652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } }
I20260812 17:50:57.462474 2652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [sys.catalog]: configured and running, proceeding with master startup.
W20260812 17:50:57.468413 2715 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:50:57.468508 2715 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:50:57.466022 2544 master_runner.cc:186] Error getting master registration for 127.13.38.62:44389: OK, instance_id { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" instance_seqno: 1786557056910710 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 17:50:57.470211 2544 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 17:50:57.474555 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.461251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51102 (local address 127.13.38.62:44389)
0812 17:50:57.461378 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.461383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.462477 (+ 1094us) server_negotiation.cc:407] Connection header received
0812 17:50:57.463627 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.463633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.463700 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.463766 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.465323 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.466076 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.467100 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.467436 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.467506 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.469463 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.469471 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.469476 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.469511 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.471837 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.472897 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.472905 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.472923 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.473032 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.473379 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.473383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.473385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.473675 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.473791 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.474215 (+ 424us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.474366 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 17:50:57.478843 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.461657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39580 (local address 127.13.38.61:36593)
0812 17:50:57.461773 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.461778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.462779 (+ 1001us) server_negotiation.cc:407] Connection header received
0812 17:50:57.462890 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.462893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.462952 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.463029 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.464143 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.464943 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.466160 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.466497 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.466597 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.471775 (+ 5178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.471782 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.471787 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.471819 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.474201 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.475862 (+ 1661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.475869 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.475899 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.476348 (+ 449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.477437 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.477441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.477442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.477699 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.477878 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.478533 (+ 655us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.478679 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 17:50:57.479017 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.462087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36638 (local address 127.13.38.60:38153)
0812 17:50:57.462202 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.462208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.463101 (+ 893us) server_negotiation.cc:407] Connection header received
0812 17:50:57.463278 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.463282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.463333 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.463387 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.466886 (+ 3499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.467636 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.469655 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.470022 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.470108 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.475434 (+ 5326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.475438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.475441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.475472 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.477048 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.477554 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.477558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.477559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.477641 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.478020 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.478024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.478025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.478253 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.478355 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.478719 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.478831 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 17:50:57.479773 2686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } } }
I20260812 17:50:57.479995 2686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:50:57.553561 2417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } attempt: 4
Aug 12 17:50:57 dist-test-slave-sb2z krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557056, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for kudu/127.13.38.60@KRBTEST.COM
I20260812 17:50:57.565311 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.554089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:38153 (local address 127.0.0.1:36648)
0812 17:50:57.554313 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 17:50:57.554329 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:50:57.554421 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:50:57.554741 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:50:57.554744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:50:57.554970 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:50:57.555088 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.555095 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.556136 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.556140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.556987 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:50:57.556998 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.557604 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.557608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:50:57.557696 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.558329 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:50:57.558345 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:50:57.560176 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:50:57.563499 (+ 3323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.563502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.563504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.563800 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.564150 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:50:57.564153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:50:57.564156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:50:57.564244 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:50:57.564759 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:50:57.564762 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:50:57.564875 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 17:50:57.564987 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 17:50:57.565138 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260812 17:50:57.565400 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:50:57.554194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36648 (local address 127.13.38.60:38153)
0812 17:50:57.554329 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 17:50:57.554334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:50:57.554379 (+ 45us) server_negotiation.cc:407] Connection header received
0812 17:50:57.554543 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:50:57.554546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:50:57.554601 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:50:57.554696 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:50:57.555231 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.556002 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.557132 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:50:57.557459 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:50:57.557586 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:50:57.560350 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:50:57.560355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:50:57.560358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:50:57.560387 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:50:57.562220 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.563930 (+ 1710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.563936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.563939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.564039 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:50:57.564354 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:50:57.564358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:50:57.564360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:50:57.564639 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:50:57.564765 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 17:50:57.565081 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 17:50:57.565232 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 17:50:57.569134 2417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8: Bootstrap starting.
I20260812 17:50:57.572203 2417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8: Neither blocks nor log segments found. Creating new log.
I20260812 17:50:57.573155 2417 log.cc:824] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8: Log is configured to *not* fsync() on all Append() calls
I20260812 17:50:57.575784 2417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8: No bootstrap required, opened a new log
I20260812 17:50:57.581373 2417 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } }
I20260812 17:50:57.581707 2417 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:50:57.581837 2417 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbfb47b2ab1148098378630270c783d8, State: Initialized, Role: FOLLOWER
I20260812 17:50:57.582412 2417 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } }
I20260812 17:50:57.583060 2718 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } } }
I20260812 17:50:57.583277 2718 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:50:57.584096 2417 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 17:50:57.590168 2738 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:50:57.590344 2738 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:50:57.596503 2309 master_runner.cc:263] No leader master found from master cbfb47b2ab1148098378630270c783d8
I20260812 17:50:57.596596 2309 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 17:50:57.636561 2465 tablet.cc:2369] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:50:57.642113 2579 tablet.cc:2369] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:50:57.657009 2341 tablet.cc:2369] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:50:57.851064 2686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:50:57.851348 2686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } }
I20260812 17:50:57.852485 2686 leader_election.cc:288] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbfb47b2ab1148098378630270c783d8 (127.13.38.62:44389), 53f625f9e53d45dbb9a43069bb539d1c (127.13.38.61:36593)
I20260812 17:50:57.853207 2382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "035082482d4b4b3b93bea3252da512ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbfb47b2ab1148098378630270c783d8" is_pre_election: true
I20260812 17:50:57.853307 2494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "035082482d4b4b3b93bea3252da512ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f625f9e53d45dbb9a43069bb539d1c" is_pre_election: true
I20260812 17:50:57.853593 2382 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 035082482d4b4b3b93bea3252da512ea in term 0.
I20260812 17:50:57.853648 2494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 035082482d4b4b3b93bea3252da512ea in term 0.
I20260812 17:50:57.854065 2573 leader_election.cc:302] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 035082482d4b4b3b93bea3252da512ea, cbfb47b2ab1148098378630270c783d8; no voters:
I20260812 17:50:57.854386 2686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:50:57.854485 2686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:50:57.854539 2686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:50:57.855638 2686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } }
I20260812 17:50:57.856047 2686 leader_election.cc:288] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [CANDIDATE]: Term 1 election: Requested vote from peers cbfb47b2ab1148098378630270c783d8 (127.13.38.62:44389), 53f625f9e53d45dbb9a43069bb539d1c (127.13.38.61:36593)
I20260812 17:50:57.856415 2382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "035082482d4b4b3b93bea3252da512ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbfb47b2ab1148098378630270c783d8"
I20260812 17:50:57.856484 2494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "035082482d4b4b3b93bea3252da512ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f625f9e53d45dbb9a43069bb539d1c"
I20260812 17:50:57.856563 2382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:50:57.856628 2494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:50:57.857930 2494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 035082482d4b4b3b93bea3252da512ea in term 1.
I20260812 17:50:57.857930 2382 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 035082482d4b4b3b93bea3252da512ea in term 1.
I20260812 17:50:57.858263 2573 leader_election.cc:302] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 035082482d4b4b3b93bea3252da512ea, cbfb47b2ab1148098378630270c783d8; no voters:
I20260812 17:50:57.858445 2686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:50:57.858705 2686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [term 1 LEADER]: Becoming Leader. State: Replica: 035082482d4b4b3b93bea3252da512ea, State: Running, Role: LEADER
I20260812 17:50:57.859095 2686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } }
I20260812 17:50:57.861153 2740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 035082482d4b4b3b93bea3252da512ea. Latest consensus state: current_term: 1 leader_uuid: "035082482d4b4b3b93bea3252da512ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } } }
I20260812 17:50:57.861284 2740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [sys.catalog]: This master's current role is: LEADER
I20260812 17:50:57.861666 2745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:50:57.862515 2745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:50:57.867470 2382 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [term 1 FOLLOWER]: Refusing update from remote peer 035082482d4b4b3b93bea3252da512ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:50:57.867640 2494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [term 1 FOLLOWER]: Refusing update from remote peer 035082482d4b4b3b93bea3252da512ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:50:57.868283 2686 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:50:57.868536 2740 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:50:57.872356 2660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 035082482d4b4b3b93bea3252da512ea. Latest consensus state: current_term: 1 leader_uuid: "035082482d4b4b3b93bea3252da512ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } } }
I20260812 17:50:57.872392 2718 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 035082482d4b4b3b93bea3252da512ea. Latest consensus state: current_term: 1 leader_uuid: "035082482d4b4b3b93bea3252da512ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } } }
I20260812 17:50:57.872498 2660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:50:57.872515 2718 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:50:57.874395 2686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "035082482d4b4b3b93bea3252da512ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } } }
I20260812 17:50:57.874536 2686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [sys.catalog]: This master's current role is: LEADER
I20260812 17:50:57.874830 2740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "035082482d4b4b3b93bea3252da512ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } } }
I20260812 17:50:57.874936 2740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea [sys.catalog]: This master's current role is: LEADER
I20260812 17:50:57.875056 2747 mvcc.cc:205] Tried to move back new op lower bound from 7317737709017931776 to 7317737708994203648. Current Snapshot: MvccSnapshot[applied={T|T < 7317737709017931776}]
I20260812 17:50:57.875206 2660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "035082482d4b4b3b93bea3252da512ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } } }
I20260812 17:50:57.875337 2660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:50:57.876168 2718 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "035082482d4b4b3b93bea3252da512ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfb47b2ab1148098378630270c783d8" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44389 } } peers { permanent_uuid: "53f625f9e53d45dbb9a43069bb539d1c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 36593 } } peers { permanent_uuid: "035082482d4b4b3b93bea3252da512ea" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 38153 } } }
I20260812 17:50:57.876296 2718 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:50:57.876597 2745 catalog_manager.cc:1347] Generated new cluster ID: 1e3f7310a3674186ab74251abf63edc3
I20260812 17:50:57.876664 2745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:50:57.887583 2745 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:50:57.888760 2745 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:50:57.905810 2745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 035082482d4b4b3b93bea3252da512ea: Generated new TSK 0
I20260812 17:50:57.906847 2745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:50:58.440868 2676 catalog_manager.cc:1259] Loaded cluster ID: 1e3f7310a3674186ab74251abf63edc3
I20260812 17:50:58.440979 2676 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c: loading cluster ID for follower catalog manager: success
I20260812 17:50:58.442639 2676 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c: acquiring CA information for follower catalog manager: success
I20260812 17:50:58.443895 2676 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 53f625f9e53d45dbb9a43069bb539d1c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 17:50:58.591861 2738 catalog_manager.cc:1259] Loaded cluster ID: 1e3f7310a3674186ab74251abf63edc3
I20260812 17:50:58.591956 2738 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8: loading cluster ID for follower catalog manager: success
I20260812 17:50:58.593608 2738 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8: acquiring CA information for follower catalog manager: success
I20260812 17:50:58.594813 2738 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cbfb47b2ab1148098378630270c783d8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 17:50:58.970453 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 2309
I20260812 17:50:58.987264 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 2427
I20260812 17:50:58.995136 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 2544
2026-08-12T17:50:59Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3741 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:50:59 dist-test-slave-sb2z krb5kdc[2766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:50:59 dist-test-slave-sb2z krb5kdc[2766](info): set up 2 sockets
Aug 12 17:50:59 dist-test-slave-sb2z krb5kdc[2766](info): commencing operation
krb5kdc: starting...
W20260812 17:51:00.286381 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:51:00 dist-test-slave-sb2z krb5kdc[2766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:51:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:51:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:51:00.427193 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34675
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:41381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:34675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:51:00.544585 2786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:00.544869 2786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:00.544911 2786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:00.549021 2786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:51:00.549090 2786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:00.549110 2786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:00.549129 2786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:51:00.549165 2786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:51:00.554677 2786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:41381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:34675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:00.555856 2786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:00.556978 2786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:00.566039 2795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:00.566061 2802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:00.566071 2798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:00.566529 2786 server_base.cc:1034] running on GCE node
I20260812 17:51:00.567198 2786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:00.568507 2786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:00.569728 2786 hybrid_clock.cc:648] HybridClock initialized: now 1786557060569712 us; error 33 us; skew 500 ppm
Aug 12 17:51:00 dist-test-slave-sb2z krb5kdc[2766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557060, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:00.572965 2786 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:51:00.574103 2786 webserver.cc:460] Webserver started at http://127.13.38.62:41925/ using document root <none> and password file <none>
I20260812 17:51:00.574709 2786 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:00.574774 2786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:00.574956 2786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:51:00.577235 2786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "0318f85ae92e4a92a44f3e692c2e7032"
format_stamp: "Formatted at 2026-08-12 17:51:00 on dist-test-slave-sb2z"
server_key: "adb0bbc7d748791a4fd0ad432d705107"
server_key_iv: "51a6e8216acf4154a7abe76a22e85059"
server_key_version: "encryptionkey@0"
I20260812 17:51:00.577798 2786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "0318f85ae92e4a92a44f3e692c2e7032"
format_stamp: "Formatted at 2026-08-12 17:51:00 on dist-test-slave-sb2z"
server_key: "adb0bbc7d748791a4fd0ad432d705107"
server_key_iv: "51a6e8216acf4154a7abe76a22e85059"
server_key_version: "encryptionkey@0"
I20260812 17:51:00.581866 2786 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:51:00.584586 2811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:00.585956 2786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:00.586154 2786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "0318f85ae92e4a92a44f3e692c2e7032"
format_stamp: "Formatted at 2026-08-12 17:51:00 on dist-test-slave-sb2z"
server_key: "adb0bbc7d748791a4fd0ad432d705107"
server_key_iv: "51a6e8216acf4154a7abe76a22e85059"
server_key_version: "encryptionkey@0"
I20260812 17:51:00.586290 2786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:00.611761 2786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:00.615382 2786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:00.615672 2786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:00.623597 2786 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:34675
I20260812 17:51:00.623667 2889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:34675 every 8 connection(s)
I20260812 17:51:00.624699 2786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:51:00.625546 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 2786
I20260812 17:51:00.625679 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:51:00.625957 13464 external_mini_cluster.cc:1428] Setting key 879a91edfd62533065fa8769075a7b2d
I20260812 17:51:00.628499 2890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:00.636224 2890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032: Bootstrap starting.
Aug 12 17:51:00 dist-test-slave-sb2z krb5kdc[2766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:00.639334 2890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:00.640411 2890 log.cc:824] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:00.642642 2893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:00.627779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40412 (local address 127.13.38.62:34675)
0812 17:51:00.628268 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:00.628289 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:00.628364 (+ 75us) server_negotiation.cc:407] Connection header received
0812 17:51:00.629263 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:00.629289 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:00.629736 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:00.630128 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:00.630912 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:00.632530 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:00.633484 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:00.634219 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:00.634386 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:00.637203 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:00.637211 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:00.637229 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:00.637267 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:00.639381 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:00.639904 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:00.639911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:00.639917 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:00.640078 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:00.640366 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:00.640370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:00.640372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:00.640982 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:00.641211 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:00.641542 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 17:51:00.641869 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":156,"threads_started":1}
I20260812 17:51:00.642805 2890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032: No bootstrap required, opened a new log
I20260812 17:51:00.645990 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:51:00.647477 2890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0318f85ae92e4a92a44f3e692c2e7032" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34675 } }
I20260812 17:51:00.647894 2890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:00.647967 2890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0318f85ae92e4a92a44f3e692c2e7032, State: Initialized, Role: FOLLOWER
I20260812 17:51:00.648458 2890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0318f85ae92e4a92a44f3e692c2e7032" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34675 } }
I20260812 17:51:00.648624 2890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:51:00.648694 2890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:51:00.648797 2890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:00.649847 2890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0318f85ae92e4a92a44f3e692c2e7032" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34675 } }
I20260812 17:51:00.650223 2890 leader_election.cc:302] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0318f85ae92e4a92a44f3e692c2e7032; no voters:
I20260812 17:51:00.650619 2890 leader_election.cc:288] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:51:00.650822 2897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:51:00.651098 2897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [term 1 LEADER]: Becoming Leader. State: Replica: 0318f85ae92e4a92a44f3e692c2e7032, State: Running, Role: LEADER
I20260812 17:51:00.651465 2897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0318f85ae92e4a92a44f3e692c2e7032" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34675 } }
I20260812 17:51:00.652060 2890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:51:00.653290 2899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0318f85ae92e4a92a44f3e692c2e7032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0318f85ae92e4a92a44f3e692c2e7032" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34675 } } }
I20260812 17:51:00.653439 2899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [sys.catalog]: This master's current role is: LEADER
I20260812 17:51:00.653743 2900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0318f85ae92e4a92a44f3e692c2e7032. Latest consensus state: current_term: 1 leader_uuid: "0318f85ae92e4a92a44f3e692c2e7032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0318f85ae92e4a92a44f3e692c2e7032" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34675 } } }
I20260812 17:51:00.653829 2911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:51:00.653863 2900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032 [sys.catalog]: This master's current role is: LEADER
I20260812 17:51:00.656845 2911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:51:00.662895 2911 catalog_manager.cc:1347] Generated new cluster ID: 4c765a97fd964a509103c21a42b769a3
I20260812 17:51:00.662994 2911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:51:00.669451 2911 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:51:00.670809 2911 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:51:00.676620 2911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0318f85ae92e4a92a44f3e692c2e7032: Generated new TSK 0
I20260812 17:51:00.677541 2911 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:51:00.702581 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 2786
2026-08-12T17:51:00Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1705 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:51:00 dist-test-slave-sb2z krb5kdc[2931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:51:00 dist-test-slave-sb2z krb5kdc[2931](info): set up 2 sockets
Aug 12 17:51:00 dist-test-slave-sb2z krb5kdc[2931](info): commencing operation
krb5kdc: starting...
W20260812 17:51:02.059861 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:51:02 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:51:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:51:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:51:02.203975 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:39111
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:39111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:51:02.331411 2952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:02.331874 2952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:02.331965 2952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:02.336149 2952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:51:02.336264 2952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:02.336301 2952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:02.336342 2952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:51:02.336396 2952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:51:02.342226 2952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:39111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:39111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:02.343463 2952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:02.344638 2952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:02.354139 2959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:02.354135 2960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:02.354101 2963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:02.354375 2952 server_base.cc:1034] running on GCE node
I20260812 17:51:02.355070 2952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:02.356331 2952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:02.357535 2952 hybrid_clock.cc:648] HybridClock initialized: now 1786557062357501 us; error 49 us; skew 500 ppm
Aug 12 17:51:02 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:02.365883 2952 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:51:02.366964 2952 webserver.cc:460] Webserver started at http://127.13.38.62:45795/ using document root <none> and password file <none>
I20260812 17:51:02.367674 2952 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:02.367753 2952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:02.367990 2952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:51:02.370254 2952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "37a852199cfc4ceaa3297db40106a8e4"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "ed0ae351233b5436f2663b28febb31cd"
server_key_iv: "c263f883160dde6ea3c48ceef6f4abda"
server_key_version: "encryptionkey@0"
I20260812 17:51:02.370879 2952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "37a852199cfc4ceaa3297db40106a8e4"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "ed0ae351233b5436f2663b28febb31cd"
server_key_iv: "c263f883160dde6ea3c48ceef6f4abda"
server_key_version: "encryptionkey@0"
I20260812 17:51:02.375463 2952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 17:51:02.378391 2974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:02.379877 2952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:02.380009 2952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "37a852199cfc4ceaa3297db40106a8e4"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "ed0ae351233b5436f2663b28febb31cd"
server_key_iv: "c263f883160dde6ea3c48ceef6f4abda"
server_key_version: "encryptionkey@0"
I20260812 17:51:02.380173 2952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:02.393822 2952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:02.397370 2952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:02.397612 2952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:02.405905 2952 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:39111
I20260812 17:51:02.406378 3043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:39111 every 8 connection(s)
I20260812 17:51:02.407297 2952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:51:02.410631 3045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:02.413321 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 2952
I20260812 17:51:02.413475 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:51:02.413797 13464 external_mini_cluster.cc:1428] Setting key c720c97b09117e1cd84c1102d4911be7
I20260812 17:51:02.417970 3045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Bootstrap starting.
I20260812 17:51:02.420827 3045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:02.421799 3045 log.cc:824] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Log is configured to *not* fsync() on all Append() calls
Aug 12 17:51:02 dist-test-slave-sb2z krb5kdc[2931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:02.424469 3045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: No bootstrap required, opened a new log
I20260812 17:51:02.427750 3045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } }
I20260812 17:51:02.428058 3045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:02.428129 3045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37a852199cfc4ceaa3297db40106a8e4, State: Initialized, Role: FOLLOWER
I20260812 17:51:02.428669 3045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } }
I20260812 17:51:02.428836 3045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:51:02.428915 3045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:51:02.429049 3045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:02.430164 3045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } }
I20260812 17:51:02.430331 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:02.415473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47574 (local address 127.13.38.62:39111)
0812 17:51:02.416007 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:02.416026 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:02.416066 (+ 40us) server_negotiation.cc:407] Connection header received
0812 17:51:02.416883 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:02.416906 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:02.417319 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:02.417626 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:02.418632 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:02.419759 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:02.421026 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:02.421550 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:02.421693 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:02.424731 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:02.424737 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:02.424752 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:02.424782 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:02.427110 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:02.427688 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:02.427695 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:02.427700 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:02.427825 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:02.428164 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:02.428170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:02.428172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:02.428638 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:02.428874 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:02.429180 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 17:51:02.429499 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":165,"threads_started":1}
I20260812 17:51:02.430616 3045 leader_election.cc:302] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37a852199cfc4ceaa3297db40106a8e4; no voters:
I20260812 17:51:02.431080 3045 leader_election.cc:288] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:51:02.431200 3054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:51:02.431439 3054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 1 LEADER]: Becoming Leader. State: Replica: 37a852199cfc4ceaa3297db40106a8e4, State: Running, Role: LEADER
I20260812 17:51:02.431790 3054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } }
I20260812 17:51:02.432477 3045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:51:02.433634 3056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37a852199cfc4ceaa3297db40106a8e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } } }
I20260812 17:51:02.433634 3059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37a852199cfc4ceaa3297db40106a8e4. Latest consensus state: current_term: 1 leader_uuid: "37a852199cfc4ceaa3297db40106a8e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } } }
I20260812 17:51:02.433802 3056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: This master's current role is: LEADER
I20260812 17:51:02.433823 3059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: This master's current role is: LEADER
I20260812 17:51:02.434223 3067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:51:02.437562 3067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:51:02.443589 3067 catalog_manager.cc:1347] Generated new cluster ID: 1f84079873c3445c817047319d4ffe34
I20260812 17:51:02.443683 3067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:51:02.464020 3067 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:51:02.465193 3067 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:51:02.477171 3067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Generated new TSK 0
I20260812 17:51:02.478032 3067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:51:02.516079 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:39111
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:51:02.650857 3082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:02.651196 3082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:02.651274 3082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:02.655444 3082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:02.655565 3082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:02.655654 3082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:51:02.661389 3082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:39111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:02.662563 3082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:02.663709 3082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:02.674304 3088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:02.674304 3090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:02.674367 3092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:02.674672 3082 server_base.cc:1034] running on GCE node
I20260812 17:51:02.675190 3082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:02.675930 3082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:02.677131 3082 hybrid_clock.cc:648] HybridClock initialized: now 1786557062677090 us; error 52 us; skew 500 ppm
Aug 12 17:51:02 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:02.680559 3082 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:51:02.681617 3082 webserver.cc:460] Webserver started at http://127.13.38.1:34805/ using document root <none> and password file <none>
I20260812 17:51:02.682240 3082 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:02.682307 3082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:02.682539 3082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:51:02.684949 3082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "dabf0cc6a8ab40049a42925dea4c2556"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "ae960c18cb070bc081904f93da7ef062"
server_key_iv: "2d29e8b4ee622f5f875f6da158ed96bf"
server_key_version: "encryptionkey@0"
I20260812 17:51:02.685626 3082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "dabf0cc6a8ab40049a42925dea4c2556"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "ae960c18cb070bc081904f93da7ef062"
server_key_iv: "2d29e8b4ee622f5f875f6da158ed96bf"
server_key_version: "encryptionkey@0"
I20260812 17:51:02.689845 3082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:51:02.692814 3102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:02.694314 3082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:02.694473 3082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "dabf0cc6a8ab40049a42925dea4c2556"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "ae960c18cb070bc081904f93da7ef062"
server_key_iv: "2d29e8b4ee622f5f875f6da158ed96bf"
server_key_version: "encryptionkey@0"
I20260812 17:51:02.694604 3082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:02.709587 3082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:02.713595 3082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:02.713857 3082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:02.714542 3082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:02.715570 3082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:51:02.715648 3082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:02.715767 3082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:51:02.715804 3082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:02.725777 3082 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:37989
I20260812 17:51:02.725798 3289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:37989 every 8 connection(s)
I20260812 17:51:02.726759 3082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:51:02.735875 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 3082
I20260812 17:51:02.736064 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:51:02.736426 13464 external_mini_cluster.cc:1428] Setting key 84bc2632e12d21eaabba65b9f054da48
Aug 12 17:51:02 dist-test-slave-sb2z krb5kdc[2931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:02.743566 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:02.730106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:59517 (local address 127.13.38.62:39111)
0812 17:51:02.730306 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:02.730314 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:02.730342 (+ 28us) server_negotiation.cc:407] Connection header received
0812 17:51:02.731631 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:02.731635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:02.731702 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:02.731796 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:02.733386 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:02.734015 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:02.735358 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:02.735712 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:02.735836 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:02.739239 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:02.739243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:02.739246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:02.739286 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:02.741075 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:02.741733 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:02.741737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:02.741739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:02.741838 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:02.742254 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:02.742259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:02.742261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:02.742486 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:02.742606 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:02.743203 (+ 597us) server_negotiation.cc:299] Negotiation successful
0812 17:51:02.743355 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 17:51:02.744782 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:02.729194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:59517)
0812 17:51:02.729730 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:02.729793 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:02.731324 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:02.731999 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:02.732010 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:02.732523 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:02.733157 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:02.733176 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:02.734196 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:02.734207 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:02.735201 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:02.735212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:02.735902 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:02.735907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:02.736215 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:02.736931 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:02.736956 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:02.739020 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:02.741232 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:02.741239 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:02.741260 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:02.741601 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:02.741956 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:02.741959 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:02.741962 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:02.742139 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:02.742621 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:02.742630 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:02.742927 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 17:51:02.743156 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 17:51:02.743594 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":116,"threads_started":1}
I20260812 17:51:02.746371 3291 heartbeater.cc:347] Connected to a master server at 127.13.38.62:39111
I20260812 17:51:02.746703 3291 heartbeater.cc:464] Registering TS with master...
I20260812 17:51:02.747465 3291 heartbeater.cc:511] Master 127.13.38.62:39111 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:51:02.749639 3000 ts_manager.cc:194] Registered new tserver with Master: dabf0cc6a8ab40049a42925dea4c2556 (127.13.38.1:37989)
I20260812 17:51:02.750952 3000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:59517
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:51:02.772153 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:39111
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:51:02.892733 3298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:02.893035 3298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:02.893082 3298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:02.897179 3298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:02.897257 3298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:02.897320 3298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:51:02.902943 3298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:39111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:02.904150 3298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:02.905169 3298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:02.915346 3313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:02.915370 3309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:02.915370 3307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:02.915663 3298 server_base.cc:1034] running on GCE node
I20260812 17:51:02.916241 3298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:02.916956 3298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:02.918179 3298 hybrid_clock.cc:648] HybridClock initialized: now 1786557062918156 us; error 34 us; skew 500 ppm
Aug 12 17:51:02 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:02.921389 3298 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:51:02.922369 3298 webserver.cc:460] Webserver started at http://127.13.38.2:43619/ using document root <none> and password file <none>
I20260812 17:51:02.923120 3298 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:02.923228 3298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:02.923485 3298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:51:02.925727 3298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "2d342d4e50ee10ccf2bfd2017aab95d2"
server_key_iv: "f86ae221390a52385c1dcef7e279febb"
server_key_version: "encryptionkey@0"
I20260812 17:51:02.926389 3298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "2d342d4e50ee10ccf2bfd2017aab95d2"
server_key_iv: "f86ae221390a52385c1dcef7e279febb"
server_key_version: "encryptionkey@0"
I20260812 17:51:02.930840 3298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 17:51:02.934021 3327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:02.935706 3298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 17:51:02.935828 3298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "2d342d4e50ee10ccf2bfd2017aab95d2"
server_key_iv: "f86ae221390a52385c1dcef7e279febb"
server_key_version: "encryptionkey@0"
I20260812 17:51:02.935935 3298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:02.950872 3298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:02.954458 3298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:02.954629 3298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:02.955224 3298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:02.956164 3298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:51:02.956218 3298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:02.956259 3298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:51:02.956276 3298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:02.965529 3298 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:34543
I20260812 17:51:02.965575 3509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:34543 every 8 connection(s)
I20260812 17:51:02.966547 3298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:51:02.970723 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 3298
I20260812 17:51:02.970885 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:51:02.971215 13464 external_mini_cluster.cc:1428] Setting key 071e07647ac43ae6d895f82b5081bff8
Aug 12 17:51:02 dist-test-slave-sb2z krb5kdc[2931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:51:02.983563 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:02.968761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:49495 (local address 127.13.38.62:39111)
0812 17:51:02.968924 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:02.968929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:02.969929 (+ 1000us) server_negotiation.cc:407] Connection header received
0812 17:51:02.971593 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:02.971597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:02.971666 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:02.971756 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:02.973242 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:02.974056 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:02.975600 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:02.975848 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:02.976005 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:02.979320 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:02.979324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:02.979326 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:02.979353 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:02.980983 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:02.981668 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:02.981672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:02.981674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:02.981771 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:02.982181 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:02.982184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:02.982186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:02.982399 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:02.982523 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:02.983191 (+ 668us) server_negotiation.cc:299] Negotiation successful
0812 17:51:02.983343 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 17:51:02.984526 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:02.969240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:49495)
0812 17:51:02.969754 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:02.969802 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:02.971260 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:02.972006 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:02.972016 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:02.972404 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:02.973064 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:02.973083 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:02.974194 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:02.974197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:02.975410 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:02.975421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:02.976057 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:02.976061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:02.976277 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:02.977047 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:02.977066 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:02.979110 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:02.981121 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:02.981128 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:02.981146 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:02.981524 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:02.981879 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:02.981883 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:02.981885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:02.982066 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:02.982523 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:02.982533 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:02.982880 (+ 347us) client_negotiation.cc:769] Sending connection context
0812 17:51:02.983138 (+ 258us) client_negotiation.cc:240] Negotiation successful
0812 17:51:02.983514 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":120,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
I20260812 17:51:02.985996 3510 heartbeater.cc:347] Connected to a master server at 127.13.38.62:39111
I20260812 17:51:02.986248 3510 heartbeater.cc:464] Registering TS with master...
I20260812 17:51:02.986747 3510 heartbeater.cc:511] Master 127.13.38.62:39111 requested a full tablet report, sending...
I20260812 17:51:02.988126 3000 ts_manager.cc:194] Registered new tserver with Master: 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543)
I20260812 17:51:02.988768 3000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:49495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:51:03.007862 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:39111
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:51:03.132426 3516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:03.132723 3516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:03.132791 3516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:03.137369 3516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:03.137450 3516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:03.137538 3516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:51:03.143416 3516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:39111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:03.144667 3516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:03.145782 3516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:03.156453 3521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:03.156510 3524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:03.156594 3516 server_base.cc:1034] running on GCE node
W20260812 17:51:03.156843 3522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:03.157310 3516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:03.158041 3516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:03.159225 3516 hybrid_clock.cc:648] HybridClock initialized: now 1786557063159173 us; error 65 us; skew 500 ppm
Aug 12 17:51:03 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557063, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:03.162505 3516 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:51:03.163627 3516 webserver.cc:460] Webserver started at http://127.13.38.3:40065/ using document root <none> and password file <none>
I20260812 17:51:03.164220 3516 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:03.164288 3516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:03.164538 3516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:51:03.166709 3516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "05b77a06b3ba4aa8a85e826945a15c70"
format_stamp: "Formatted at 2026-08-12 17:51:03 on dist-test-slave-sb2z"
server_key: "02983ed2dd083a08f275a2bd8af070bf"
server_key_iv: "7172d10438cfce8453fbbed3a8ebe2ab"
server_key_version: "encryptionkey@0"
I20260812 17:51:03.167300 3516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "05b77a06b3ba4aa8a85e826945a15c70"
format_stamp: "Formatted at 2026-08-12 17:51:03 on dist-test-slave-sb2z"
server_key: "02983ed2dd083a08f275a2bd8af070bf"
server_key_iv: "7172d10438cfce8453fbbed3a8ebe2ab"
server_key_version: "encryptionkey@0"
I20260812 17:51:03.171377 3516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 17:51:03.174292 3535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:03.175895 3516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:03.176021 3516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "05b77a06b3ba4aa8a85e826945a15c70"
format_stamp: "Formatted at 2026-08-12 17:51:03 on dist-test-slave-sb2z"
server_key: "02983ed2dd083a08f275a2bd8af070bf"
server_key_iv: "7172d10438cfce8453fbbed3a8ebe2ab"
server_key_version: "encryptionkey@0"
I20260812 17:51:03.176128 3516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:03.201352 3516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:03.204778 3516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:03.204954 3516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:03.205549 3516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:03.206461 3516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:51:03.206512 3516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:03.206557 3516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:51:03.206573 3516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:03.216471 3516 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:40785
I20260812 17:51:03.216529 3715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:40785 every 8 connection(s)
I20260812 17:51:03.217465 3516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:51:03.226841 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 3516
I20260812 17:51:03.227022 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:51:03.227319 13464 external_mini_cluster.cc:1428] Setting key 28b214f8f7221022d85f8897a0da5a95
Aug 12 17:51:03 dist-test-slave-sb2z krb5kdc[2931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557063, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:03.232265 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.219454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:49069 (local address 127.13.38.62:39111)
0812 17:51:03.219622 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:03.219626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:03.220544 (+ 918us) server_negotiation.cc:407] Connection header received
0812 17:51:03.222014 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:03.222017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:03.222071 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:03.222149 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:03.223373 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.223940 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.225115 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.225334 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.225419 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.228121 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:03.228125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:03.228127 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:03.228158 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:03.229934 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:03.230477 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:03.230481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:03.230483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:03.230572 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:03.230964 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:03.230967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:03.230968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:03.231213 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:03.231298 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:03.232013 (+ 715us) server_negotiation.cc:299] Negotiation successful
0812 17:51:03.232136 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 17:51:03.233204 3720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.219865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:49069)
0812 17:51:03.220366 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:03.220411 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:03.221779 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:03.222306 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:03.222313 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:03.222685 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:03.223194 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:03.223208 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.224078 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.224081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:03.224980 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:03.224990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.225477 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.225480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:03.225627 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.226209 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:03.226225 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:03.227941 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:03.230070 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:03.230075 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:03.230090 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:03.230371 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:03.230668 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:03.230671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:03.230672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:03.230882 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:03.231343 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:03.231349 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:03.231719 (+ 370us) client_negotiation.cc:769] Sending connection context
0812 17:51:03.231910 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 17:51:03.232225 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":124,"threads_started":1}
I20260812 17:51:03.234743 3716 heartbeater.cc:347] Connected to a master server at 127.13.38.62:39111
I20260812 17:51:03.235051 3716 heartbeater.cc:464] Registering TS with master...
I20260812 17:51:03.235713 3716 heartbeater.cc:511] Master 127.13.38.62:39111 requested a full tablet report, sending...
I20260812 17:51:03.236889 3000 ts_manager.cc:194] Registered new tserver with Master: 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785)
I20260812 17:51:03.237468 3000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:49069
I20260812 17:51:03.241583 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:51:03.251822 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.243762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47582 (local address 127.13.38.62:39111)
0812 17:51:03.243912 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:03.243915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:03.243968 (+ 53us) server_negotiation.cc:407] Connection header received
0812 17:51:03.244077 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:03.244079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:03.244121 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:03.244188 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:03.244885 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.245425 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.246503 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.246732 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.246825 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.248356 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:03.248359 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:03.248361 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:03.248387 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:03.249938 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:03.250403 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:03.250406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:03.250407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:03.250498 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:03.250819 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:03.250822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:03.250823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:03.251124 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:03.251235 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:03.251457 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 17:51:03.251624 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 17:51:03.255438 3000 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:51:03.258706 3000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:51:03.275494 3742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.269097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60132 (local address 127.13.38.2:34543)
0812 17:51:03.269527 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:03.269546 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:03.270206 (+ 660us) server_negotiation.cc:407] Connection header received
0812 17:51:03.270529 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:03.270534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:03.270737 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:03.270866 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:03.271441 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.272297 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.273813 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.274636 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.274757 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.274927 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:03.275167 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 17:51:03.275285 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":91,"threads_started":1}
I20260812 17:51:03.275599 3745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.269641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.13.38.1:37989)
0812 17:51:03.269948 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:03.269954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:03.270205 (+ 251us) server_negotiation.cc:407] Connection header received
0812 17:51:03.270527 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:03.270533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:03.270737 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:03.270866 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:03.271415 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.272597 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.274129 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.274922 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.275059 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.275203 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:03.275299 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 17:51:03.275392 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
I20260812 17:51:03.275583 3737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.269461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:37989 (local address 127.0.0.1:47064)
0812 17:51:03.270080 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:03.270109 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:03.270340 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:03.271063 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:03.271066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:03.271095 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:03.271287 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:03.271294 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.272721 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.272724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:03.273973 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:03.273985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.275063 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.275067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:03.275165 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.275189 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 17:51:03.275267 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 17:51:03.275409 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":56,"spinlock_wait_cycles":50560,"thread_start_us":102,"threads_started":1}
I20260812 17:51:03.275791 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.268838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56328 (local address 127.13.38.3:40785)
0812 17:51:03.269666 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:03.269676 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:03.270201 (+ 525us) server_negotiation.cc:407] Connection header received
0812 17:51:03.270529 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:03.270534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:03.270708 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:03.270876 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:03.271206 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.272507 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.274246 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.274886 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.274992 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.275167 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:03.275493 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 17:51:03.275627 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":193,"threads_started":1}
I20260812 17:51:03.276206 3744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.268638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:34543 (local address 127.0.0.1:60132)
0812 17:51:03.270052 (+ 1414us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:03.270110 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:03.270340 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:03.271063 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:03.271066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:03.271094 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:03.271288 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:03.271294 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.272462 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.272465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:03.273683 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:03.273693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.274786 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.274789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:03.274904 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.274969 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 17:51:03.275774 (+ 805us) client_negotiation.cc:240] Negotiation successful
0812 17:51:03.276024 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":56,"spinlock_wait_cycles":648704,"thread_start_us":90,"threads_started":1}
I20260812 17:51:03.276491 3738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.268638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:40785 (local address 127.0.0.1:56328)
0812 17:51:03.270051 (+ 1413us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:03.270111 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:03.270340 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:03.270870 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:03.270876 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:03.270901 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:03.271058 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:03.271069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.272688 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.272694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:03.274103 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:03.274111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.275237 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.275241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:03.275321 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.275343 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 17:51:03.276174 (+ 831us) client_negotiation.cc:240] Negotiation successful
0812 17:51:03.276345 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":237,"spinlock_wait_cycles":24192,"thread_start_us":125,"threads_started":1}
I20260812 17:51:03.279186 3403 tablet_service.cc:1467] Processing CreateTablet for tablet c19468daf6ae40fbaa20e9ee1245f4b4 (DEFAULT_TABLE table=test-table [id=698f6c235b0b4a7b96b59330fd79b8ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:51:03.279203 3177 tablet_service.cc:1467] Processing CreateTablet for tablet c19468daf6ae40fbaa20e9ee1245f4b4 (DEFAULT_TABLE table=test-table [id=698f6c235b0b4a7b96b59330fd79b8ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:51:03.279211 3616 tablet_service.cc:1467] Processing CreateTablet for tablet c19468daf6ae40fbaa20e9ee1245f4b4 (DEFAULT_TABLE table=test-table [id=698f6c235b0b4a7b96b59330fd79b8ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:51:03.280431 3177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c19468daf6ae40fbaa20e9ee1245f4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:03.280437 3616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c19468daf6ae40fbaa20e9ee1245f4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:03.280452 3403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c19468daf6ae40fbaa20e9ee1245f4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:03.288512 3749 tablet_bootstrap.cc:492] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70: Bootstrap starting.
I20260812 17:51:03.288777 3746 tablet_bootstrap.cc:492] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Bootstrap starting.
I20260812 17:51:03.288955 3748 tablet_bootstrap.cc:492] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Bootstrap starting.
I20260812 17:51:03.290959 3749 tablet_bootstrap.cc:654] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:03.290959 3748 tablet_bootstrap.cc:654] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:03.291263 3746 tablet_bootstrap.cc:654] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:03.292084 3746 log.cc:824] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:03.292062 3748 log.cc:824] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:03.292302 3749 log.cc:824] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:03.294520 3746 tablet_bootstrap.cc:492] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c: No bootstrap required, opened a new log
I20260812 17:51:03.294523 3748 tablet_bootstrap.cc:492] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: No bootstrap required, opened a new log
I20260812 17:51:03.294549 3749 tablet_bootstrap.cc:492] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70: No bootstrap required, opened a new log
I20260812 17:51:03.294747 3748 ts_tablet_manager.cc:1397] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:51:03.294749 3746 ts_tablet_manager.cc:1397] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 17:51:03.294770 3749 ts_tablet_manager.cc:1397] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 17:51:03.297860 3749 raft_consensus.cc:348] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:03.298135 3749 raft_consensus.cc:374] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:03.298193 3749 raft_consensus.cc:729] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b77a06b3ba4aa8a85e826945a15c70, State: Initialized, Role: FOLLOWER
I20260812 17:51:03.298686 3749 consensus_queue.cc:260] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:03.299008 3746 raft_consensus.cc:348] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:03.299278 3746 raft_consensus.cc:374] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:03.299340 3746 raft_consensus.cc:729] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d84ffefdb3746e4a0971ebd6c98bd4c, State: Initialized, Role: FOLLOWER
I20260812 17:51:03.299716 3716 heartbeater.cc:503] Master 127.13.38.62:39111 was elected leader, sending a full tablet report...
I20260812 17:51:03.299885 3749 ts_tablet_manager.cc:1428] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 17:51:03.299875 3746 consensus_queue.cc:260] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:03.300799 3510 heartbeater.cc:503] Master 127.13.38.62:39111 was elected leader, sending a full tablet report...
I20260812 17:51:03.300927 3746 ts_tablet_manager.cc:1428] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 17:51:03.299537 3748 raft_consensus.cc:348] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:03.302744 3748 raft_consensus.cc:374] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:03.302809 3748 raft_consensus.cc:729] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dabf0cc6a8ab40049a42925dea4c2556, State: Initialized, Role: FOLLOWER
I20260812 17:51:03.303296 3748 consensus_queue.cc:260] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:03.304087 3291 heartbeater.cc:503] Master 127.13.38.62:39111 was elected leader, sending a full tablet report...
I20260812 17:51:03.305596 3748 ts_tablet_manager.cc:1428] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Time spent starting tablet: real 0.011s user 0.005s sys 0.003s
W20260812 17:51:03.467824 3511 tablet.cc:2369] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:51:03.468560 3718 tablet.cc:2369] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:51:03.478277 3292 tablet.cc:2369] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:51:03.582758 3759 raft_consensus.cc:482] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:51:03.582996 3759 raft_consensus.cc:504] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:03.584352 3759 leader_election.cc:288] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543), dabf0cc6a8ab40049a42925dea4c2556 (127.13.38.1:37989)
I20260812 17:51:03.589656 3745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.584912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:50075 (local address 127.13.38.1:37989)
0812 17:51:03.585089 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:03.585093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:03.585268 (+ 175us) server_negotiation.cc:407] Connection header received
0812 17:51:03.585419 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:03.585422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:03.585474 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:03.585545 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:03.585845 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.586562 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.588281 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.588920 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.589080 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.589138 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:03.589433 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 17:51:03.589512 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 17:51:03.589656 3742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.584767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:49221 (local address 127.13.38.2:34543)
0812 17:51:03.584973 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:03.584978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:03.584998 (+ 20us) server_negotiation.cc:407] Connection header received
0812 17:51:03.585110 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:03.585115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:03.585171 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:03.585279 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:03.585670 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.586434 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.588281 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.588916 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.589039 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.589122 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:03.589433 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 17:51:03.589530 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260812 17:51:03.589689 3720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.584655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:34543 (local address 127.13.38.3:49221)
0812 17:51:03.584855 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:03.584873 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:03.584991 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:03.585299 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:03.585303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:03.585321 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:03.585499 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:03.585507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.586582 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.586585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:03.588123 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:03.588136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.589068 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.589074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:03.589244 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.589321 (+ 77us) client_negotiation.cc:769] Sending connection context
0812 17:51:03.589403 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 17:51:03.589522 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":138}
I20260812 17:51:03.589699 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.584677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:37989 (local address 127.13.38.3:50075)
0812 17:51:03.585167 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:03.585179 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:03.585299 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:03.585598 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:03.585602 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:03.585627 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:03.585736 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:03.585741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.586683 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.586686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:03.588142 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:03.588150 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.589105 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.589109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:03.589270 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.589312 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 17:51:03.589403 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 17:51:03.589522 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":56,"thread_start_us":122,"threads_started":1}
I20260812 17:51:03.590544 3430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c19468daf6ae40fbaa20e9ee1245f4b4" candidate_uuid: "05b77a06b3ba4aa8a85e826945a15c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" is_pre_election: true
I20260812 17:51:03.590590 3212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c19468daf6ae40fbaa20e9ee1245f4b4" candidate_uuid: "05b77a06b3ba4aa8a85e826945a15c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabf0cc6a8ab40049a42925dea4c2556" is_pre_election: true
I20260812 17:51:03.590898 3212 raft_consensus.cc:2454] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b77a06b3ba4aa8a85e826945a15c70 in term 0.
I20260812 17:51:03.590893 3430 raft_consensus.cc:2454] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b77a06b3ba4aa8a85e826945a15c70 in term 0.
I20260812 17:51:03.591439 3536 leader_election.cc:302] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b77a06b3ba4aa8a85e826945a15c70, dabf0cc6a8ab40049a42925dea4c2556; no voters:
I20260812 17:51:03.591804 3759 raft_consensus.cc:2781] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:51:03.591918 3759 raft_consensus.cc:482] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:51:03.591972 3759 raft_consensus.cc:3037] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:03.593146 3759 raft_consensus.cc:504] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:03.593560 3759 leader_election.cc:288] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [CANDIDATE]: Term 1 election: Requested vote from peers 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543), dabf0cc6a8ab40049a42925dea4c2556 (127.13.38.1:37989)
I20260812 17:51:03.594000 3430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c19468daf6ae40fbaa20e9ee1245f4b4" candidate_uuid: "05b77a06b3ba4aa8a85e826945a15c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c"
I20260812 17:51:03.594156 3430 raft_consensus.cc:3037] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:03.594247 3212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c19468daf6ae40fbaa20e9ee1245f4b4" candidate_uuid: "05b77a06b3ba4aa8a85e826945a15c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabf0cc6a8ab40049a42925dea4c2556"
I20260812 17:51:03.594426 3212 raft_consensus.cc:3037] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:03.595360 3430 raft_consensus.cc:2454] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b77a06b3ba4aa8a85e826945a15c70 in term 1.
I20260812 17:51:03.595582 3212 raft_consensus.cc:2454] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b77a06b3ba4aa8a85e826945a15c70 in term 1.
I20260812 17:51:03.595731 3540 leader_election.cc:302] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b77a06b3ba4aa8a85e826945a15c70, 8d84ffefdb3746e4a0971ebd6c98bd4c; no voters:
I20260812 17:51:03.595906 3759 raft_consensus.cc:2781] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:51:03.596151 3759 raft_consensus.cc:686] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 1 LEADER]: Becoming Leader. State: Replica: 05b77a06b3ba4aa8a85e826945a15c70, State: Running, Role: LEADER
I20260812 17:51:03.596539 3759 consensus_queue.cc:237] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:03.600210 3000 catalog_manager.cc:5495] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3). New cstate: current_term: 1 leader_uuid: "05b77a06b3ba4aa8a85e826945a15c70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } health_report { overall_health: UNKNOWN } } }
I20260812 17:51:03.645953 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.642066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56336 (local address 127.13.38.3:40785)
0812 17:51:03.642235 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:03.642241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:03.642258 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:51:03.642383 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:03.642387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:03.642459 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:03.642585 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:51:03.642858 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.643409 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.644339 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:03.644584 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:03.644701 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:03.645047 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:51:03.645493 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:51:03.645636 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:03.645700 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 17:51:03.645759 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 17:51:03.652413 3430 raft_consensus.cc:1261] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 1 FOLLOWER]: Refusing update from remote peer 05b77a06b3ba4aa8a85e826945a15c70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:51:03.652413 3212 raft_consensus.cc:1261] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Refusing update from remote peer 05b77a06b3ba4aa8a85e826945a15c70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:51:03.653193 3759 consensus_queue.cc:1035] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:51:03.653450 3766 consensus_queue.cc:1035] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:51:03.663692 3771 mvcc.cc:205] Tried to move back new op lower bound from 7317737732711862272 to 7317737732494594048. Current Snapshot: MvccSnapshot[applied={T|T < 7317737732711862272}]
I20260812 17:51:03.670359 2999 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:51:03.670591 2999 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:51:03.673457 2999 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Sending DeleteTablet for 3 replicas of tablet c19468daf6ae40fbaa20e9ee1245f4b4
I20260812 17:51:03.674445 3616 tablet_service.cc:1514] Processing DeleteTablet for tablet c19468daf6ae40fbaa20e9ee1245f4b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:03 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:56328
I20260812 17:51:03.674470 3403 tablet_service.cc:1514] Processing DeleteTablet for tablet c19468daf6ae40fbaa20e9ee1245f4b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:03 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:60132
I20260812 17:51:03.674613 3177 tablet_service.cc:1514] Processing DeleteTablet for tablet c19468daf6ae40fbaa20e9ee1245f4b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:03 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:47064
I20260812 17:51:03.674973 3781 tablet_replica.cc:330] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70: stopping tablet replica
I20260812 17:51:03.675251 3786 tablet_replica.cc:330] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: stopping tablet replica
I20260812 17:51:03.675289 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 3082
I20260812 17:51:03.675330 3781 raft_consensus.cc:2229] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:51:03.675722 3786 raft_consensus.cc:2229] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:51:03.679545 3782 tablet_replica.cc:330] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c: stopping tablet replica
I20260812 17:51:03.679939 3782 raft_consensus.cc:2229] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:51:03.680342 3782 raft_consensus.cc:2258] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:51:03.684324 3782 ts_tablet_manager.cc:1905] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:51:03.685807 3781 raft_consensus.cc:2258] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:51:03.687492 3782 ts_tablet_manager.cc:1918] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:51:03.687650 3782 log.cc:1196] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/c19468daf6ae40fbaa20e9ee1245f4b4
I20260812 17:51:03.687808 3781 ts_tablet_manager.cc:1905] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:51:03.688014 3782 ts_tablet_manager.cc:1939] T c19468daf6ae40fbaa20e9ee1245f4b4 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Deleting consensus metadata
I20260812 17:51:03.689527 2979 catalog_manager.cc:4841] TS 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543): tablet c19468daf6ae40fbaa20e9ee1245f4b4 (table test-table [id=698f6c235b0b4a7b96b59330fd79b8ad]) successfully deleted
I20260812 17:51:03.691154 3781 ts_tablet_manager.cc:1918] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:51:03.691296 3781 log.cc:1196] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/c19468daf6ae40fbaa20e9ee1245f4b4
I20260812 17:51:03.691660 3781 ts_tablet_manager.cc:1939] T c19468daf6ae40fbaa20e9ee1245f4b4 P 05b77a06b3ba4aa8a85e826945a15c70: Deleting consensus metadata
I20260812 17:51:03.692883 2976 catalog_manager.cc:4841] TS 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785): tablet c19468daf6ae40fbaa20e9ee1245f4b4 (table test-table [id=698f6c235b0b4a7b96b59330fd79b8ad]) successfully deleted
W20260812 17:51:03.698406 2975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:37989 (error 108)
I20260812 17:51:03.699115 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 3298
I20260812 17:51:03.700049 3744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:03.699327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:37989 (local address 127.0.0.1:47068)
0812 17:51:03.699806 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:03.699904 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362}
W20260812 17:51:03.701767 2975 catalog_manager.cc:4584] TS dabf0cc6a8ab40049a42925dea4c2556 (127.13.38.1:37989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c19468daf6ae40fbaa20e9ee1245f4b4: Network error: Client connection negotiation failed: client connection to 127.13.38.1:37989: connect: Connection refused (error 111)
I20260812 17:51:03.717144 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 3516
I20260812 17:51:03.730059 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 2952
I20260812 17:51:03.745951 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:39111
--webserver_interface=127.13.38.62
--webserver_port=45795
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:39111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:51:03.866940 3788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:03.867256 3788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:03.867300 3788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:03.871515 3788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:51:03.871665 3788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:03.871686 3788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:03.871704 3788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:51:03.871719 3788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:51:03.877403 3788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:39111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:39111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=45795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:03.878648 3788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:03.879837 3788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:03.895862 3800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:03.895833 3796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:03.896013 3797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:03.896204 3788 server_base.cc:1034] running on GCE node
I20260812 17:51:03.896960 3788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:03.898252 3788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:03.899479 3788 hybrid_clock.cc:648] HybridClock initialized: now 1786557063899460 us; error 44 us; skew 500 ppm
Aug 12 17:51:03 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557063, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:03.902660 3788 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:51:03.903924 3788 webserver.cc:460] Webserver started at http://127.13.38.62:45795/ using document root <none> and password file <none>
I20260812 17:51:03.904569 3788 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:03.904655 3788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:03.908788 3788 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 17:51:03.911131 3814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:03.912699 3788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:51:03.912824 3788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "37a852199cfc4ceaa3297db40106a8e4"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "ed0ae351233b5436f2663b28febb31cd"
server_key_iv: "c263f883160dde6ea3c48ceef6f4abda"
server_key_version: "encryptionkey@0"
I20260812 17:51:03.913535 3788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:03.928736 3788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:03.931901 3788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:03.932081 3788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:03.939728 3788 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:39111
I20260812 17:51:03.940043 3890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:39111 every 8 connection(s)
I20260812 17:51:03.941083 3788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:51:03.944133 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 3788
I20260812 17:51:03.944614 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:37989
--local_ip_for_outbound_sockets=127.13.38.1
--tserver_master_addrs=127.13.38.62:39111
--webserver_port=34805
--webserver_interface=127.13.38.1
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:51:03.946403 3891 sys_catalog.cc:263] Verifying existing consensus state
I20260812 17:51:03.949162 3891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Bootstrap starting.
I20260812 17:51:03.974814 3891 log.cc:824] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:03.988215 3891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 17:51:03.988781 3891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Bootstrap complete.
I20260812 17:51:03.995270 3891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } }
I20260812 17:51:03.995942 3891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37a852199cfc4ceaa3297db40106a8e4, State: Initialized, Role: FOLLOWER
I20260812 17:51:03.996585 3891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } }
I20260812 17:51:03.996755 3891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:51:03.996842 3891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:51:03.996961 3891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 1 FOLLOWER]: Advancing to term 2
I20260812 17:51:04.000053 3891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } }
I20260812 17:51:04.000509 3891 leader_election.cc:302] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37a852199cfc4ceaa3297db40106a8e4; no voters:
I20260812 17:51:04.000974 3891 leader_election.cc:288] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 17:51:04.001247 3896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 17:51:04.001694 3896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 2 LEADER]: Becoming Leader. State: Replica: 37a852199cfc4ceaa3297db40106a8e4, State: Running, Role: LEADER
I20260812 17:51:04.002065 3896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } }
I20260812 17:51:04.002540 3891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:51:04.003805 3898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37a852199cfc4ceaa3297db40106a8e4. Latest consensus state: current_term: 2 leader_uuid: "37a852199cfc4ceaa3297db40106a8e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } } }
I20260812 17:51:04.003950 3898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: This master's current role is: LEADER
I20260812 17:51:04.004347 3897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "37a852199cfc4ceaa3297db40106a8e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } } }
I20260812 17:51:04.004468 3897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: This master's current role is: LEADER
I20260812 17:51:04.005303 3912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:51:04.009801 3912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:51:04.010448 3912 catalog_manager.cc:1259] Loaded cluster ID: 1f84079873c3445c817047319d4ffe34
I20260812 17:51:04.010552 3912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:51:04.012022 3912 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 17:51:04.015182 3912 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 17:51:04.020402 3912 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:51:04.021304 3912 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Loaded TSK: 0
I20260812 17:51:04.022387 3912 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 17:51:04.099751 3894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:04.100068 3894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:04.100165 3894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:04.104678 3894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:04.104781 3894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:04.104872 3894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:51:04.110330 3894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:37989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=34805
--tserver_master_addrs=127.13.38.62:39111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:04.111474 3894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:04.112560 3894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:04.123014 3944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:04.123005 3945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:04.123317 3894 server_base.cc:1034] running on GCE node
W20260812 17:51:04.123396 3948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:04.123817 3894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:04.124550 3894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:04.125771 3894 hybrid_clock.cc:648] HybridClock initialized: now 1786557064125731 us; error 47 us; skew 500 ppm
Aug 12 17:51:04 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557064, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:04.128814 3894 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:51:04.129881 3894 webserver.cc:460] Webserver started at http://127.13.38.1:34805/ using document root <none> and password file <none>
I20260812 17:51:04.130494 3894 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:04.130582 3894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:04.135039 3894 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:51:04.137481 3960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:04.138898 3894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:51:04.139006 3894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "dabf0cc6a8ab40049a42925dea4c2556"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "ae960c18cb070bc081904f93da7ef062"
server_key_iv: "2d29e8b4ee622f5f875f6da158ed96bf"
server_key_version: "encryptionkey@0"
I20260812 17:51:04.139674 3894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:04.168784 3894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:04.172452 3894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:04.172706 3894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:04.173378 3894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:04.174575 3968 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 17:51:04.179185 3894 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 17:51:04.179302 3894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260812 17:51:04.179399 3894 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 17:51:04.181351 3894 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 17:51:04.181437 3894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 17:51:04.181789 3968 tablet_bootstrap.cc:492] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Bootstrap starting.
I20260812 17:51:04.192472 3894 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:37989
I20260812 17:51:04.192567 4126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:37989 every 8 connection(s)
I20260812 17:51:04.193516 3894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:51:04.194939 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 3894
I20260812 17:51:04.195446 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:34543
--local_ip_for_outbound_sockets=127.13.38.2
--tserver_master_addrs=127.13.38.62:39111
--webserver_port=43619
--webserver_interface=127.13.38.2
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:51:04.203706 3968 log.cc:824] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Log is configured to *not* fsync() on all Append() calls
Aug 12 17:51:04 dist-test-slave-sb2z krb5kdc[2931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557064, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:04.213860 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.195720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:54493 (local address 127.13.38.62:39111)
0812 17:51:04.196490 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:04.196512 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:04.196973 (+ 461us) server_negotiation.cc:407] Connection header received
0812 17:51:04.198595 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:04.198620 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:04.199164 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:04.199610 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:04.201274 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.202764 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.205000 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.205568 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.205773 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.208433 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:04.208439 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:04.208452 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:04.208483 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:04.210323 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:04.210914 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:04.210919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:04.210922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:04.211058 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:04.211431 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:04.211440 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:04.211442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:04.211879 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:04.212049 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:04.212680 (+ 631us) server_negotiation.cc:299] Negotiation successful
0812 17:51:04.212951 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":133,"threads_started":1}
I20260812 17:51:04.213888 4135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.195754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:54493)
0812 17:51:04.196797 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:04.196841 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:04.198041 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:04.199725 (+ 1684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:04.199738 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:04.200295 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:04.201031 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.201066 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.202979 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.202986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.204826 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.204845 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.205739 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.205743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.205937 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.206606 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:04.206623 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:04.208202 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:04.210466 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:04.210472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:04.210486 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:04.210776 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:04.211169 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:04.211172 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:04.211174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:04.211322 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:04.212103 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:04.212108 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:04.212350 (+ 242us) client_negotiation.cc:769] Sending connection context
0812 17:51:04.212539 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 17:51:04.212891 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"thread_start_us":150,"threads_started":1}
I20260812 17:51:04.214381 3968 tablet_bootstrap.cc:492] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 17:51:04.214854 3968 tablet_bootstrap.cc:492] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Bootstrap complete.
I20260812 17:51:04.216596 3968 ts_tablet_manager.cc:1397] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Time spent bootstrapping tablet: real 0.035s user 0.024s sys 0.008s
I20260812 17:51:04.217038 4127 heartbeater.cc:347] Connected to a master server at 127.13.38.62:39111
I20260812 17:51:04.217295 4127 heartbeater.cc:464] Registering TS with master...
I20260812 17:51:04.217909 4127 heartbeater.cc:511] Master 127.13.38.62:39111 requested a full tablet report, sending...
I20260812 17:51:04.219954 3968 raft_consensus.cc:348] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:04.220511 3968 raft_consensus.cc:729] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dabf0cc6a8ab40049a42925dea4c2556, State: Initialized, Role: FOLLOWER
I20260812 17:51:04.220610 3839 ts_manager.cc:194] Registered new tserver with Master: dabf0cc6a8ab40049a42925dea4c2556 (127.13.38.1:37989)
I20260812 17:51:04.221045 3968 consensus_queue.cc:260] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:04.222173 3968 ts_tablet_manager.cc:1428] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 17:51:04.224500 3839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:54493
I20260812 17:51:04.226830 4127 heartbeater.cc:503] Master 127.13.38.62:39111 was elected leader, sending a full tablet report...
I20260812 17:51:04.232450 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.223977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47082 (local address 127.13.38.1:37989)
0812 17:51:04.227748 (+ 3771us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:04.227761 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:04.227784 (+ 23us) server_negotiation.cc:407] Connection header received
0812 17:51:04.227855 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:04.227861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:04.228087 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:04.228268 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:04.228610 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.229676 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.230873 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.231721 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.231843 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.231988 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:04.232183 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 17:51:04.232288 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3669,"thread_start_us":121,"threads_started":1}
I20260812 17:51:04.232467 4140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.223881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:37989 (local address 127.0.0.1:47082)
0812 17:51:04.227355 (+ 3474us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:04.227406 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:04.227583 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:04.228288 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:04.228294 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:04.228323 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:04.228463 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.228470 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.229819 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.229822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.230724 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.230736 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.231875 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.231878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.231977 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.232013 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 17:51:04.232121 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 17:51:04.232300 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3274,"thread_start_us":144,"threads_started":1}
I20260812 17:51:04.233888 4031 tablet_service.cc:1514] Processing DeleteTablet for tablet c19468daf6ae40fbaa20e9ee1245f4b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:03 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:47082
I20260812 17:51:04.234874 4145 tablet_replica.cc:330] stopping tablet replica
I20260812 17:51:04.235240 4145 raft_consensus.cc:2229] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:51:04.235515 4145 raft_consensus.cc:2258] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:51:04.236683 4145 ts_tablet_manager.cc:1905] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:51:04.240481 4145 ts_tablet_manager.cc:1918] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:51:04.240578 4145 log.cc:1196] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal/wals/c19468daf6ae40fbaa20e9ee1245f4b4
I20260812 17:51:04.240866 4145 ts_tablet_manager.cc:1939] T c19468daf6ae40fbaa20e9ee1245f4b4 P dabf0cc6a8ab40049a42925dea4c2556: Deleting consensus metadata
I20260812 17:51:04.243058 3815 catalog_manager.cc:4841] TS dabf0cc6a8ab40049a42925dea4c2556 (127.13.38.1:37989): tablet c19468daf6ae40fbaa20e9ee1245f4b4 (table test-table [id=698f6c235b0b4a7b96b59330fd79b8ad]) successfully deleted
W20260812 17:51:04.326483 4137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:04.326824 4137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:04.326908 4137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:04.331182 4137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:04.331288 4137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:04.331387 4137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:51:04.337072 4137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:34543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=43619
--tserver_master_addrs=127.13.38.62:39111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:04.338294 4137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:04.339423 4137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:04.349922 4152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:04.349938 4156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:04.349982 4153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:04.350407 4137 server_base.cc:1034] running on GCE node
I20260812 17:51:04.350862 4137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:04.351678 4137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:04.352898 4137 hybrid_clock.cc:648] HybridClock initialized: now 1786557064352880 us; error 29 us; skew 500 ppm
Aug 12 17:51:04 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557064, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:04.356045 4137 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:51:04.357034 4137 webserver.cc:460] Webserver started at http://127.13.38.2:43619/ using document root <none> and password file <none>
I20260812 17:51:04.357645 4137 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:04.357729 4137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:04.362078 4137 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 17:51:04.364456 4166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:04.365856 4137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:51:04.366027 4137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "2d342d4e50ee10ccf2bfd2017aab95d2"
server_key_iv: "f86ae221390a52385c1dcef7e279febb"
server_key_version: "encryptionkey@0"
I20260812 17:51:04.366659 4137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:04.397538 4137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:04.400956 4137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:04.401197 4137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:04.401856 4137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:04.402829 4137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:51:04.402905 4137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:04.402987 4137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:51:04.403029 4137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:04.412757 4137 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:34543
I20260812 17:51:04.412781 4344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:34543 every 8 connection(s)
I20260812 17:51:04.413847 4137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:51:04.415912 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 4137
I20260812 17:51:04.416389 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:40785
--local_ip_for_outbound_sockets=127.13.38.3
--tserver_master_addrs=127.13.38.62:39111
--webserver_port=40065
--webserver_interface=127.13.38.3
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 17:51:04 dist-test-slave-sb2z krb5kdc[2931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557064, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:04.429116 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.415824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:39425 (local address 127.13.38.62:39111)
0812 17:51:04.415999 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:04.416004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:04.417127 (+ 1123us) server_negotiation.cc:407] Connection header received
0812 17:51:04.418718 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:04.418725 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:04.418795 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:04.418910 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:04.420203 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.420737 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.422045 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.422292 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.422404 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.425085 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:04.425088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:04.425090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:04.425120 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:04.426757 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:04.427333 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:04.427336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:04.427337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:04.427420 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:04.427846 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:04.427850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:04.427853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:04.428069 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:04.428184 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:04.428743 (+ 559us) server_negotiation.cc:299] Negotiation successful
0812 17:51:04.428890 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 17:51:04.429934 4351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.416306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:39425)
0812 17:51:04.416951 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:04.417006 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:04.418473 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:04.419086 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:04.419093 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:04.419472 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:04.420022 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.420036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.420893 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.420897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.421902 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.421913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.422466 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.422471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.422647 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.423284 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:04.423301 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:04.424885 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:04.426887 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:04.426894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:04.426911 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:04.427205 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:04.427551 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:04.427563 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:04.427568 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:04.427731 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:04.428184 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:04.428192 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:04.428471 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 17:51:04.428681 (+ 210us) client_negotiation.cc:240] Negotiation successful
0812 17:51:04.429038 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":143,"threads_started":1}
I20260812 17:51:04.431170 4347 heartbeater.cc:347] Connected to a master server at 127.13.38.62:39111
I20260812 17:51:04.431450 4347 heartbeater.cc:464] Registering TS with master...
I20260812 17:51:04.432036 4347 heartbeater.cc:511] Master 127.13.38.62:39111 requested a full tablet report, sending...
I20260812 17:51:04.433225 3839 ts_manager.cc:194] Registered new tserver with Master: 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543)
I20260812 17:51:04.433845 3839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:39425
W20260812 17:51:04.540741 4353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:04.541122 4353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:04.541240 4353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:04.545820 4353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:04.545903 4353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:04.545991 4353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:51:04.551510 4353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:40785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=40065
--tserver_master_addrs=127.13.38.62:39111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:04.552698 4353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:04.553716 4353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 17:51:04.564376 4353 server_base.cc:1034] running on GCE node
W20260812 17:51:04.564304 4358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:04.564394 4360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:04.564296 4363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:04.565021 4353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:04.565833 4353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:04.567044 4353 hybrid_clock.cc:648] HybridClock initialized: now 1786557064567005 us; error 50 us; skew 500 ppm
Aug 12 17:51:04 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557064, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:04.570201 4353 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:51:04.571228 4353 webserver.cc:460] Webserver started at http://127.13.38.3:40065/ using document root <none> and password file <none>
I20260812 17:51:04.571930 4353 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:04.572021 4353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:04.576579 4353 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:51:04.578984 4375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:04.580312 4353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:04.580447 4353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "05b77a06b3ba4aa8a85e826945a15c70"
format_stamp: "Formatted at 2026-08-12 17:51:03 on dist-test-slave-sb2z"
server_key: "02983ed2dd083a08f275a2bd8af070bf"
server_key_iv: "7172d10438cfce8453fbbed3a8ebe2ab"
server_key_version: "encryptionkey@0"
I20260812 17:51:04.581027 4353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:04.599633 4353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:04.603142 4353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:04.603438 4353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:04.604185 4353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:04.605144 4353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:51:04.605195 4353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:04.605259 4353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:51:04.605293 4353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:04.614494 4353 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:40785
I20260812 17:51:04.614519 4564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:40785 every 8 connection(s)
I20260812 17:51:04.615648 4353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:51:04.625470 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 4353
Aug 12 17:51:04 dist-test-slave-sb2z krb5kdc[2931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557064, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:04.630668 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.617798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:53303 (local address 127.13.38.62:39111)
0812 17:51:04.617984 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:04.617989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:04.618969 (+ 980us) server_negotiation.cc:407] Connection header received
0812 17:51:04.620426 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:04.620429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:04.620483 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:04.620564 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:04.621898 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.622444 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.623618 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.623836 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.623953 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.626632 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:04.626635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:04.626638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:04.626679 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:04.628398 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:04.628972 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:04.628975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:04.628976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:04.629056 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:04.629460 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:04.629463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:04.629464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:04.629633 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:04.629707 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:04.630338 (+ 631us) server_negotiation.cc:299] Negotiation successful
0812 17:51:04.630457 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 17:51:04.631858 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.618260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:53303)
0812 17:51:04.618748 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:04.618795 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:04.620161 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:04.620749 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:04.620757 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:04.621195 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:04.621743 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.621757 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.622561 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.622564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.623452 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.623461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.623977 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.623979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.624128 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.624679 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:04.624697 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:04.626382 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:04.628529 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:04.628535 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:04.628554 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:04.628868 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:04.629150 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:04.629153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:04.629156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:04.629353 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:04.629741 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:04.629748 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:04.630071 (+ 323us) client_negotiation.cc:769] Sending connection context
0812 17:51:04.630296 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 17:51:04.630663 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":135,"threads_started":1}
I20260812 17:51:04.633241 4565 heartbeater.cc:347] Connected to a master server at 127.13.38.62:39111
I20260812 17:51:04.633545 4565 heartbeater.cc:464] Registering TS with master...
I20260812 17:51:04.634079 4565 heartbeater.cc:511] Master 127.13.38.62:39111 requested a full tablet report, sending...
I20260812 17:51:04.635217 3839 ts_manager.cc:194] Registered new tserver with Master: 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785)
I20260812 17:51:04.635653 4571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.626247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47586 (local address 127.13.38.62:39111)
0812 17:51:04.626531 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:04.626535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:04.626694 (+ 159us) server_negotiation.cc:407] Connection header received
0812 17:51:04.626797 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:04.626800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:04.626853 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:04.627010 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:04.628030 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.628768 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.629856 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.630086 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.630161 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.631744 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:04.631748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:04.631751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:04.631782 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:04.633416 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:04.633896 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:04.633901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:04.633903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:04.633989 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:04.634298 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:04.634301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:04.634303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:04.634582 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:04.634799 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:04.635315 (+ 516us) server_negotiation.cc:299] Negotiation successful
0812 17:51:04.635462 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":93,"threads_started":1}
I20260812 17:51:04.636006 3839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:53303
I20260812 17:51:04.637526 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:51:04.642354 4571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.638191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47596 (local address 127.13.38.62:39111)
0812 17:51:04.638322 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:04.638328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:04.638642 (+ 314us) server_negotiation.cc:407] Connection header received
0812 17:51:04.638726 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:04.638728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:04.638826 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:04.638907 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:51:04.639146 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.639751 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.640821 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.641053 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.641157 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.641589 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:51:04.641966 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:51:04.642098 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:04.642196 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 17:51:04.642251 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 17:51:04.643132 3839 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:51:04.646529 3839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:51:04.655550 4031 tablet_service.cc:1467] Processing CreateTablet for tablet b92eecbdd8dd474a9e3ff61451347131 (DEFAULT_TABLE table=test-table [id=629af6abc3b84f77bc7fb59789476c77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:51:04.656304 4031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92eecbdd8dd474a9e3ff61451347131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:04.659462 3968 tablet_bootstrap.cc:492] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556: Bootstrap starting.
I20260812 17:51:04.659889 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.654594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:34543 (local address 127.0.0.1:60140)
0812 17:51:04.655189 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:04.655207 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:04.655326 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:04.656069 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:04.656073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:04.656095 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:04.656189 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.656195 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.657514 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.657517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.658622 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.658630 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.659435 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.659439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.659555 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.659586 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 17:51:04.659652 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 17:51:04.659727 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":137,"threads_started":1}
I20260812 17:51:04.660115 4140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.654210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:40785 (local address 127.0.0.1:56342)
0812 17:51:04.654343 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:04.654361 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:04.654449 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:04.655853 (+ 1404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:04.655856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:04.655874 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:04.656005 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.656030 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.657316 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.657320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.658525 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.658534 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.659718 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.659722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.659819 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.659845 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 17:51:04.659910 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 17:51:04.659990 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 17:51:04.660295 4581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.655063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60140 (local address 127.13.38.2:34543)
0812 17:51:04.655552 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:04.655569 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:04.655597 (+ 28us) server_negotiation.cc:407] Connection header received
0812 17:51:04.655682 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:04.655688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:04.655904 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:04.656094 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:04.656302 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.657377 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.658717 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.659301 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.659439 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.659743 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:04.659806 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 17:51:04.659896 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":138,"threads_started":1}
I20260812 17:51:04.660511 3968 tablet_bootstrap.cc:654] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:04.661455 4579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.654615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56342 (local address 127.13.38.3:40785)
0812 17:51:04.655218 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:04.655236 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:04.655265 (+ 29us) server_negotiation.cc:407] Connection header received
0812 17:51:04.655367 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:04.655376 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:04.655670 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:04.656187 (+ 517us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:04.656352 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.657163 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.658669 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.659574 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.660750 (+ 1176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.660950 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:04.661169 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 17:51:04.661286 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":161,"threads_started":1}
I20260812 17:51:04.661847 3968 tablet_bootstrap.cc:492] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556: No bootstrap required, opened a new log
I20260812 17:51:04.661921 3968 ts_tablet_manager.cc:1397] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:51:04.662238 3968 raft_consensus.cc:348] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:04.662384 3968 raft_consensus.cc:374] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:04.662439 3968 raft_consensus.cc:729] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dabf0cc6a8ab40049a42925dea4c2556, State: Initialized, Role: FOLLOWER
I20260812 17:51:04.662565 3968 consensus_queue.cc:260] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:04.662825 3968 ts_tablet_manager.cc:1428] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 17:51:04.663694 4242 tablet_service.cc:1467] Processing CreateTablet for tablet b92eecbdd8dd474a9e3ff61451347131 (DEFAULT_TABLE table=test-table [id=629af6abc3b84f77bc7fb59789476c77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:51:04.664588 4459 tablet_service.cc:1467] Processing CreateTablet for tablet b92eecbdd8dd474a9e3ff61451347131 (DEFAULT_TABLE table=test-table [id=629af6abc3b84f77bc7fb59789476c77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:51:04.665416 4242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92eecbdd8dd474a9e3ff61451347131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:04.665702 4459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92eecbdd8dd474a9e3ff61451347131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:04.673309 4587 tablet_bootstrap.cc:492] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70: Bootstrap starting.
I20260812 17:51:04.674612 4589 tablet_bootstrap.cc:492] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Bootstrap starting.
I20260812 17:51:04.675462 4587 tablet_bootstrap.cc:654] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:04.676656 4587 log.cc:824] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:04.676995 4589 tablet_bootstrap.cc:654] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:04.677724 4589 log.cc:824] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:04.678634 4587 tablet_bootstrap.cc:492] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70: No bootstrap required, opened a new log
I20260812 17:51:04.678850 4587 ts_tablet_manager.cc:1397] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:51:04.679813 4589 tablet_bootstrap.cc:492] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c: No bootstrap required, opened a new log
I20260812 17:51:04.680025 4589 ts_tablet_manager.cc:1397] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 17:51:04.683156 4587 raft_consensus.cc:348] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:04.683396 4587 raft_consensus.cc:374] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:04.683434 4587 raft_consensus.cc:729] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b77a06b3ba4aa8a85e826945a15c70, State: Initialized, Role: FOLLOWER
I20260812 17:51:04.683305 4589 raft_consensus.cc:348] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:04.683708 4589 raft_consensus.cc:374] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:04.683784 4589 raft_consensus.cc:729] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d84ffefdb3746e4a0971ebd6c98bd4c, State: Initialized, Role: FOLLOWER
I20260812 17:51:04.683974 4587 consensus_queue.cc:260] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:04.684320 4589 consensus_queue.cc:260] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:04.684756 4565 heartbeater.cc:503] Master 127.13.38.62:39111 was elected leader, sending a full tablet report...
I20260812 17:51:04.685127 4587 ts_tablet_manager.cc:1428] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 17:51:04.685420 4589 ts_tablet_manager.cc:1428] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 17:51:04.685676 4347 heartbeater.cc:503] Master 127.13.38.62:39111 was elected leader, sending a full tablet report...
W20260812 17:51:04.695212 4129 tablet.cc:2369] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:51:04.719776 4595 raft_consensus.cc:482] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:51:04.720026 4595 raft_consensus.cc:504] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:04.721366 4595 leader_election.cc:288] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785), dabf0cc6a8ab40049a42925dea4c2556 (127.13.38.1:37989)
I20260812 17:51:04.726454 4351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.721654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:37989 (local address 127.13.38.2:44413)
0812 17:51:04.721913 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:04.721944 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:04.722049 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:04.722361 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:04.722365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:04.722385 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:04.722524 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.722530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.723636 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.723640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.725206 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.725222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.725946 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.725950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.726069 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.726096 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 17:51:04.726204 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 17:51:04.726289 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":83}
I20260812 17:51:04.726457 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.721739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:44413 (local address 127.13.38.1:37989)
0812 17:51:04.721886 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:04.721891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:04.722011 (+ 120us) server_negotiation.cc:407] Connection header received
0812 17:51:04.722178 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:04.722181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:04.722233 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:04.722346 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:04.722654 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.723510 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.725335 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.725797 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.725932 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.726007 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:04.726225 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 17:51:04.726309 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 17:51:04.726756 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.721683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:40785 (local address 127.13.38.2:40477)
0812 17:51:04.722098 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:04.722114 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:04.722209 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:04.722470 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:04.722473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:04.722490 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:04.722618 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.722623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.723586 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.723593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.725185 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:04.725198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.726296 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.726299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:04.726404 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.726447 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 17:51:04.726532 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 17:51:04.726607 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":87,"thread_start_us":162,"threads_started":1}
I20260812 17:51:04.726743 4579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.721786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:40477 (local address 127.13.38.3:40785)
0812 17:51:04.721947 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:04.721952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:04.722197 (+ 245us) server_negotiation.cc:407] Connection header received
0812 17:51:04.722298 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:04.722301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:04.722356 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:04.722440 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:04.722734 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.723423 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.725351 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.726017 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.726113 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.726180 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:04.726555 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 17:51:04.726633 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 17:51:04.727227 4058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92eecbdd8dd474a9e3ff61451347131" candidate_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabf0cc6a8ab40049a42925dea4c2556" is_pre_election: true
I20260812 17:51:04.727634 4058 raft_consensus.cc:2454] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d84ffefdb3746e4a0971ebd6c98bd4c in term 0.
I20260812 17:51:04.727632 4495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92eecbdd8dd474a9e3ff61451347131" candidate_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b77a06b3ba4aa8a85e826945a15c70" is_pre_election: true
I20260812 17:51:04.728003 4495 raft_consensus.cc:2454] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d84ffefdb3746e4a0971ebd6c98bd4c in term 0.
I20260812 17:51:04.728147 4167 leader_election.cc:302] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d84ffefdb3746e4a0971ebd6c98bd4c, dabf0cc6a8ab40049a42925dea4c2556; no voters:
I20260812 17:51:04.728487 4595 raft_consensus.cc:2781] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:51:04.728605 4595 raft_consensus.cc:482] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:51:04.728658 4595 raft_consensus.cc:3037] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:04.729992 4595 raft_consensus.cc:504] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:04.730413 4595 leader_election.cc:288] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [CANDIDATE]: Term 1 election: Requested vote from peers 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785), dabf0cc6a8ab40049a42925dea4c2556 (127.13.38.1:37989)
I20260812 17:51:04.730774 4495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92eecbdd8dd474a9e3ff61451347131" candidate_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b77a06b3ba4aa8a85e826945a15c70"
I20260812 17:51:04.730950 4058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92eecbdd8dd474a9e3ff61451347131" candidate_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dabf0cc6a8ab40049a42925dea4c2556"
I20260812 17:51:04.730993 4495 raft_consensus.cc:3037] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:04.731104 4058 raft_consensus.cc:3037] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:04.732250 4495 raft_consensus.cc:2454] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d84ffefdb3746e4a0971ebd6c98bd4c in term 1.
I20260812 17:51:04.732329 4058 raft_consensus.cc:2454] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d84ffefdb3746e4a0971ebd6c98bd4c in term 1.
I20260812 17:51:04.732625 4167 leader_election.cc:302] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d84ffefdb3746e4a0971ebd6c98bd4c, dabf0cc6a8ab40049a42925dea4c2556; no voters:
I20260812 17:51:04.732827 4595 raft_consensus.cc:2781] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:51:04.733078 4595 raft_consensus.cc:686] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 1 LEADER]: Becoming Leader. State: Replica: 8d84ffefdb3746e4a0971ebd6c98bd4c, State: Running, Role: LEADER
I20260812 17:51:04.733424 4595 consensus_queue.cc:237] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:04.736799 3838 catalog_manager.cc:5495] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2). New cstate: current_term: 1 leader_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } health_report { overall_health: UNKNOWN } } }
I20260812 17:51:04.764720 4581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.760908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60144 (local address 127.13.38.2:34543)
0812 17:51:04.761065 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:04.761070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:04.761086 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:51:04.761270 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:04.761272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:04.761332 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:04.761463 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:51:04.761717 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.762234 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.763082 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:04.763308 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:04.763425 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:04.763815 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:51:04.764246 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:51:04.764390 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:04.764454 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 17:51:04.764513 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 17:51:04.770875 4495 raft_consensus.cc:1261] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70 [term 1 FOLLOWER]: Refusing update from remote peer 8d84ffefdb3746e4a0971ebd6c98bd4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:51:04.770918 4058 raft_consensus.cc:1261] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Refusing update from remote peer 8d84ffefdb3746e4a0971ebd6c98bd4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:51:04.771764 4597 consensus_queue.cc:1035] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:51:04.771996 4595 consensus_queue.cc:1035] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:51:04.779956 4601 mvcc.cc:205] Tried to move back new op lower bound from 7317737737293864960 to 7317737737151123456. Current Snapshot: MvccSnapshot[applied={T|T < 7317737737293864960}]
I20260812 17:51:04.789088 3838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:51:04.789309 3838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:51:04.792187 3838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Sending DeleteTablet for 3 replicas of tablet b92eecbdd8dd474a9e3ff61451347131
I20260812 17:51:04.793054 4031 tablet_service.cc:1514] Processing DeleteTablet for tablet b92eecbdd8dd474a9e3ff61451347131 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:04 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:47082
I20260812 17:51:04.793164 4242 tablet_service.cc:1514] Processing DeleteTablet for tablet b92eecbdd8dd474a9e3ff61451347131 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:04 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:60140
I20260812 17:51:04.793709 4615 tablet_replica.cc:330] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c: stopping tablet replica
I20260812 17:51:04.793992 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 3894
I20260812 17:51:04.794001 4459 tablet_service.cc:1514] Processing DeleteTablet for tablet b92eecbdd8dd474a9e3ff61451347131 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:04 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:56342
I20260812 17:51:04.802206 4615 raft_consensus.cc:2229] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:51:04.803115 4615 raft_consensus.cc:2258] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:51:04.804868 4617 tablet_replica.cc:330] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70: stopping tablet replica
I20260812 17:51:04.805311 4617 raft_consensus.cc:2229] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:51:04.805707 4617 raft_consensus.cc:2258] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:51:04.805964 4615 ts_tablet_manager.cc:1905] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:51:04.807518 4617 ts_tablet_manager.cc:1905] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:51:04.810385 4615 ts_tablet_manager.cc:1918] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:51:04.810418 4617 ts_tablet_manager.cc:1918] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:51:04.810535 4615 log.cc:1196] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/b92eecbdd8dd474a9e3ff61451347131
I20260812 17:51:04.810544 4617 log.cc:1196] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/b92eecbdd8dd474a9e3ff61451347131
I20260812 17:51:04.810921 4617 ts_tablet_manager.cc:1939] T b92eecbdd8dd474a9e3ff61451347131 P 05b77a06b3ba4aa8a85e826945a15c70: Deleting consensus metadata
I20260812 17:51:04.810954 4615 ts_tablet_manager.cc:1939] T b92eecbdd8dd474a9e3ff61451347131 P 8d84ffefdb3746e4a0971ebd6c98bd4c: Deleting consensus metadata
I20260812 17:51:04.812414 3819 catalog_manager.cc:4841] TS 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543): tablet b92eecbdd8dd474a9e3ff61451347131 (table test-table [id=629af6abc3b84f77bc7fb59789476c77]) successfully deleted
I20260812 17:51:04.812417 3816 catalog_manager.cc:4841] TS 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785): tablet b92eecbdd8dd474a9e3ff61451347131 (table test-table [id=629af6abc3b84f77bc7fb59789476c77]) successfully deleted
W20260812 17:51:04.817060 3815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:37989 (error 108)
I20260812 17:51:04.817704 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 4137
I20260812 17:51:04.821947 4140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:04.817982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:37989 (local address 127.0.0.1:47092)
0812 17:51:04.821664 (+ 3682us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:04.821789 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3540}
W20260812 17:51:04.825037 3815 catalog_manager.cc:4584] TS dabf0cc6a8ab40049a42925dea4c2556 (127.13.38.1:37989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b92eecbdd8dd474a9e3ff61451347131: Network error: Client connection negotiation failed: client connection to 127.13.38.1:37989: connect: Connection refused (error 111)
I20260812 17:51:04.844646 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 4353
I20260812 17:51:04.857290 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 3788
I20260812 17:51:04.874073 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:39111
--webserver_interface=127.13.38.62
--webserver_port=45795
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:39111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:51:04.993975 4619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:04.994297 4619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:04.994341 4619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:04.998533 4619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:51:04.998611 4619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:04.998631 4619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:04.998648 4619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:51:04.998664 4619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:51:05.004583 4619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:39111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:39111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=45795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:05.005791 4619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:05.006897 4619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:05.015941 4629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:05.015974 4625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:05.015946 4626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:05.016472 4619 server_base.cc:1034] running on GCE node
I20260812 17:51:05.017200 4619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:05.018530 4619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:05.019721 4619 hybrid_clock.cc:648] HybridClock initialized: now 1786557065019665 us; error 71 us; skew 500 ppm
Aug 12 17:51:05 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557065, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:05.023221 4619 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:51:05.024330 4619 webserver.cc:460] Webserver started at http://127.13.38.62:45795/ using document root <none> and password file <none>
I20260812 17:51:05.024940 4619 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:05.025012 4619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:05.029485 4619 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 17:51:05.032058 4640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:05.033623 4619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:05.033746 4619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "37a852199cfc4ceaa3297db40106a8e4"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "ed0ae351233b5436f2663b28febb31cd"
server_key_iv: "c263f883160dde6ea3c48ceef6f4abda"
server_key_version: "encryptionkey@0"
I20260812 17:51:05.034327 4619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:05.049518 4619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:05.053371 4619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:05.053551 4619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:05.061611 4619 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:39111
I20260812 17:51:05.061930 4721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:39111 every 8 connection(s)
I20260812 17:51:05.062883 4619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:51:05.067811 4723 sys_catalog.cc:263] Verifying existing consensus state
I20260812 17:51:05.069653 4723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Bootstrap starting.
I20260812 17:51:05.072552 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 4619
I20260812 17:51:05.073047 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:37989
--local_ip_for_outbound_sockets=127.13.38.1
--tserver_master_addrs=127.13.38.62:39111
--webserver_port=34805
--webserver_interface=127.13.38.1
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:51:05.088577 4723 log.cc:824] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:05.100787 4723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 17:51:05.101248 4723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4: Bootstrap complete.
I20260812 17:51:05.105526 4723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } }
I20260812 17:51:05.105979 4723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37a852199cfc4ceaa3297db40106a8e4, State: Initialized, Role: FOLLOWER
I20260812 17:51:05.106550 4723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } }
I20260812 17:51:05.106711 4723 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:51:05.106803 4723 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:51:05.106935 4723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 2 FOLLOWER]: Advancing to term 3
I20260812 17:51:05.109066 4723 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } }
I20260812 17:51:05.109431 4723 leader_election.cc:302] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37a852199cfc4ceaa3297db40106a8e4; no voters:
I20260812 17:51:05.109828 4723 leader_election.cc:288] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 17:51:05.109920 4734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 17:51:05.110136 4734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [term 3 LEADER]: Becoming Leader. State: Replica: 37a852199cfc4ceaa3297db40106a8e4, State: Running, Role: LEADER
I20260812 17:51:05.110461 4734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } }
I20260812 17:51:05.111184 4723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:51:05.111888 4736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "37a852199cfc4ceaa3297db40106a8e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } } }
I20260812 17:51:05.112030 4736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: This master's current role is: LEADER
I20260812 17:51:05.112373 4735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37a852199cfc4ceaa3297db40106a8e4. Latest consensus state: current_term: 3 leader_uuid: "37a852199cfc4ceaa3297db40106a8e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a852199cfc4ceaa3297db40106a8e4" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39111 } } }
I20260812 17:51:05.112476 4735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37a852199cfc4ceaa3297db40106a8e4 [sys.catalog]: This master's current role is: LEADER
I20260812 17:51:05.113039 4740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:51:05.118369 4740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:51:05.119081 4740 catalog_manager.cc:1259] Loaded cluster ID: 1f84079873c3445c817047319d4ffe34
I20260812 17:51:05.119272 4740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 17:51:05.120595 4740 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f16ba5ee9ad google::LogMessage::Fail() at ??:0
@ 0x7f16ba5f188a google::LogMessage::SendToLog() at ??:0
@ 0x7f16ba5ee4de google::LogMessage::Flush() at ??:0
@ 0x7f16ba5efa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f16c4f31fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f16c4f32d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f16c4f2f579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f16c4f73640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f16bc8403d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f16bc840c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f16bc8423f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f16bc831512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f16bde756db start_thread at ??:0
@ 0x7f16b90ec61f clone at ??:0
W20260812 17:51:05.217113 4727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:05.217466 4727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:05.217547 4727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:05.221938 4727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:05.222038 4727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:05.222131 4727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:51:05.227834 4727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:37989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=34805
--tserver_master_addrs=127.13.38.62:39111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:05.229081 4727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:05.230345 4727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:05.242591 4790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:05.242712 4793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:05.242554 4789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:05.243005 4727 server_base.cc:1034] running on GCE node
I20260812 17:51:05.243619 4727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:05.244354 4727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:05.245548 4727 hybrid_clock.cc:648] HybridClock initialized: now 1786557065245519 us; error 49 us; skew 500 ppm
Aug 12 17:51:05 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557065, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:05.248586 4727 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:51:05.249591 4727 webserver.cc:460] Webserver started at http://127.13.38.1:34805/ using document root <none> and password file <none>
I20260812 17:51:05.250207 4727 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:05.250273 4727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:05.254565 4727 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:51:05.257100 4804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:05.258651 4727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 17:51:05.258807 4727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "dabf0cc6a8ab40049a42925dea4c2556"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "ae960c18cb070bc081904f93da7ef062"
server_key_iv: "2d29e8b4ee622f5f875f6da158ed96bf"
server_key_version: "encryptionkey@0"
I20260812 17:51:05.259413 4727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:05.273777 4727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:05.277529 4727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:05.277765 4727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:05.278470 4727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:05.279749 4814 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 17:51:05.284049 4727 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 17:51:05.284116 4727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260812 17:51:05.284166 4727 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 17:51:05.285904 4727 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 17:51:05.285974 4727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 17:51:05.286242 4814 tablet_bootstrap.cc:492] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556: Bootstrap starting.
I20260812 17:51:05.297180 4727 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:37989
I20260812 17:51:05.298113 4986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:37989 every 8 connection(s)
I20260812 17:51:05.298571 4727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:51:05.301816 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 4727
I20260812 17:51:05.302341 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:34543
--local_ip_for_outbound_sockets=127.13.38.2
--tserver_master_addrs=127.13.38.62:39111
--webserver_port=43619
--webserver_interface=127.13.38.2
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:51:05.309005 4814 log.cc:824] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:05.316448 4814 tablet_bootstrap.cc:492] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 17:51:05.317106 4814 tablet_bootstrap.cc:492] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556: Bootstrap complete.
I20260812 17:51:05.318984 4814 ts_tablet_manager.cc:1397] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556: Time spent bootstrapping tablet: real 0.033s user 0.019s sys 0.012s
I20260812 17:51:05.323005 4814 raft_consensus.cc:348] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:05.323748 4814 raft_consensus.cc:729] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dabf0cc6a8ab40049a42925dea4c2556, State: Initialized, Role: FOLLOWER
I20260812 17:51:05.324422 4814 consensus_queue.cc:260] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:05.325779 4814 ts_tablet_manager.cc:1428] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260812 17:51:05.393275 4989 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:51:05.301583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:53853)
0812 17:51:05.303076 (+ 1493us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:05.303148 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:05.304398 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:05.392404 (+ 88006us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1126,"thread_start_us":180,"threads_started":1}
W20260812 17:51:05.393846 4987 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:39111 (0 consecutive failures): Network error: Failed to ping master at 127.13.38.62:39111: Client connection negotiation failed: client connection to 127.13.38.62:39111: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 17:51:05.394709 4989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:05.394418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:35975)
0812 17:51:05.394569 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:05.394637 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 17:51:05.395408 4989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:05.395190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:50493)
0812 17:51:05.395308 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:05.395360 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 17:51:05.395607 4987 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 17:51:05.461977 4991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:05.462337 4991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:05.462443 4991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:05.467221 4991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:05.467331 4991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:05.467443 4991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:51:05.473557 4991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:34543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=43619
--tserver_master_addrs=127.13.38.62:39111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:05.474807 4991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:05.476027 4991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:05.486943 5001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:05.486991 5003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:05.486945 5006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:05.487272 4991 server_base.cc:1034] running on GCE node
I20260812 17:51:05.487854 4991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:05.488611 4991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:05.489811 4991 hybrid_clock.cc:648] HybridClock initialized: now 1786557065489773 us; error 49 us; skew 500 ppm
Aug 12 17:51:05 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557065, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:05.493132 4991 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:51:05.494239 4991 webserver.cc:460] Webserver started at http://127.13.38.2:43619/ using document root <none> and password file <none>
I20260812 17:51:05.494885 4991 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:05.494995 4991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:05.499415 4991 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:05.502017 5018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:05.503720 4991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:05.503895 4991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c"
format_stamp: "Formatted at 2026-08-12 17:51:02 on dist-test-slave-sb2z"
server_key: "2d342d4e50ee10ccf2bfd2017aab95d2"
server_key_iv: "f86ae221390a52385c1dcef7e279febb"
server_key_version: "encryptionkey@0"
I20260812 17:51:05.504567 4991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:05.541568 4991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:05.545449 4991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:05.545686 4991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:05.546365 4991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:05.547283 4991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:51:05.547358 4991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:05.547433 4991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:51:05.547487 4991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:05.557224 4991 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:34543
I20260812 17:51:05.557245 5210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:34543 every 8 connection(s)
I20260812 17:51:05.558354 4991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:51:05.561156 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 4991
I20260812 17:51:05.561689 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:40785
--local_ip_for_outbound_sockets=127.13.38.3
--tserver_master_addrs=127.13.38.62:39111
--webserver_port=40065
--webserver_interface=127.13.38.3
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:51:05.563513 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:05.560781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:55119)
0812 17:51:05.561384 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:05.561555 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":131,"threads_started":1}
W20260812 17:51:05.564256 5211 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:39111 (0 consecutive failures): Network error: Failed to ping master at 127.13.38.62:39111: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
I20260812 17:51:05.565126 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:05.564817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:52933)
0812 17:51:05.564958 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:05.565013 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 17:51:05.565837 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:05.565614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:43841)
0812 17:51:05.565730 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:05.565784 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 17:51:05.566026 5211 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 17:51:05.684687 5214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:05.685035 5214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:05.685134 5214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:05.689366 5214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:05.689481 5214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:05.689584 5214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:51:05.695254 5214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:40785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=40065
--tserver_master_addrs=127.13.38.62:39111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:05.696514 5214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:05.697664 5214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:05.708159 5226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:05.708159 5223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:05.708269 5222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:05.708735 5214 server_base.cc:1034] running on GCE node
I20260812 17:51:05.709142 5214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:05.709859 5214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:05.711030 5214 hybrid_clock.cc:648] HybridClock initialized: now 1786557065711005 us; error 61 us; skew 500 ppm
Aug 12 17:51:05 dist-test-slave-sb2z krb5kdc[2931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557065, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:05.717926 5214 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:51:05.719157 5214 webserver.cc:460] Webserver started at http://127.13.38.3:40065/ using document root <none> and password file <none>
I20260812 17:51:05.719905 5214 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:05.720011 5214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:05.724615 5214 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260812 17:51:05.727173 5242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:05.729311 5214 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 17:51:05.729467 5214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "05b77a06b3ba4aa8a85e826945a15c70"
format_stamp: "Formatted at 2026-08-12 17:51:03 on dist-test-slave-sb2z"
server_key: "02983ed2dd083a08f275a2bd8af070bf"
server_key_iv: "7172d10438cfce8453fbbed3a8ebe2ab"
server_key_version: "encryptionkey@0"
I20260812 17:51:05.730317 5214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:05.751361 5214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:05.755149 5214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:05.755403 5214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:05.756261 5214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:05.757198 5214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:51:05.757287 5214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:05.757362 5214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:51:05.757414 5214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:05.767747 5214 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:40785
I20260812 17:51:05.767831 5418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:40785 every 8 connection(s)
I20260812 17:51:05.768951 5214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:51:05.771677 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 5214
I20260812 17:51:05.774057 5421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:05.771805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:36447)
0812 17:51:05.772436 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:05.772593 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":170,"threads_started":1}
W20260812 17:51:05.774585 5419 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:39111 (0 consecutive failures): Network error: Failed to ping master at 127.13.38.62:39111: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
I20260812 17:51:05.775378 5421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:05.775087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:37627)
0812 17:51:05.775250 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:05.775311 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260812 17:51:05.776096 5421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:05.775868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:47513)
0812 17:51:05.775983 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:05.776031 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 17:51:05.776298 5419 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 17:51:06.396849 5423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:06.396341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:52407)
0812 17:51:06.396656 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:06.396723 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":100,"threads_started":1}
I20260812 17:51:06.567199 5427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:06.566705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:43415)
0812 17:51:06.567003 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:06.567081 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":84,"threads_started":1}
I20260812 17:51:06.608608 5428 raft_consensus.cc:482] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:51:06.608820 5428 raft_consensus.cc:504] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:06.610091 5428 leader_election.cc:288] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785), 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543)
Aug 12 17:51:06 dist-test-slave-sb2z krb5kdc[2931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557065, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.2@KRBTEST.COM
Aug 12 17:51:06 dist-test-slave-sb2z krb5kdc[2931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557065, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.3@KRBTEST.COM
I20260812 17:51:06.626766 5434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:06.610360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:34543 (local address 127.13.38.1:57815)
0812 17:51:06.611501 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:06.611546 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:06.611664 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:06.613522 (+ 1858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:06.613525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:06.613817 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:06.614347 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:06.614363 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:06.615770 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:06.615773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:06.616848 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:06.616861 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:06.617650 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:06.617654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:06.617854 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:06.618544 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:06.618568 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:06.620579 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:06.623985 (+ 3406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:06.623990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:06.623993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:06.624424 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:06.624828 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:06.624831 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:06.624833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:06.624998 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:06.625718 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:06.625726 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:06.625967 (+ 241us) client_negotiation.cc:769] Sending connection context
0812 17:51:06.626179 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 17:51:06.626541 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1057,"mutex_wait_us":814,"spinlock_wait_cycles":35072,"thread_start_us":71,"threads_started":1}
I20260812 17:51:06.626782 5429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:06.610490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:57815 (local address 127.13.38.2:34543)
0812 17:51:06.610793 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:06.610818 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:06.611654 (+ 836us) server_negotiation.cc:407] Connection header received
0812 17:51:06.612531 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:06.612539 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:06.612901 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:06.613183 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:06.614550 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:06.615621 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:06.617013 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:06.617510 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:06.617705 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:06.620913 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:06.620986 (+ 73us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:06.621012 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:06.621061 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:06.623851 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:06.624576 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:06.624581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:06.624585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:06.624715 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:06.625108 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:06.625112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:06.625114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:06.625585 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:06.625744 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:06.626323 (+ 579us) server_negotiation.cc:299] Negotiation successful
0812 17:51:06.626613 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":93,"threads_started":1}
I20260812 17:51:06.627101 5431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:06.610767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:54051 (local address 127.13.38.3:40785)
0812 17:51:06.611182 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:06.611202 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:06.611237 (+ 35us) server_negotiation.cc:407] Connection header received
0812 17:51:06.612141 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:06.612151 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:06.612655 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:06.613003 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:06.614590 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:06.616351 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:06.617600 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:06.618434 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:06.618638 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:06.621612 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:06.621618 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:06.621630 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:06.621662 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:06.623736 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:06.624543 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:06.624553 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:06.624561 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:06.624746 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:06.625137 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:06.625143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:06.625148 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:06.625740 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:06.625870 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:06.626483 (+ 613us) server_negotiation.cc:299] Negotiation successful
0812 17:51:06.626821 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":137,"threads_started":1}
I20260812 17:51:06.627458 5423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:06.610344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:40785 (local address 127.13.38.1:54051)
0812 17:51:06.610827 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:06.610844 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:06.610984 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:06.613126 (+ 2142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:06.613135 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:06.613821 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:06.614350 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:06.614370 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:06.616545 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:06.616551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:06.617415 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:06.617430 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:06.618579 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:06.618582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:06.618680 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:06.619243 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:06.619264 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:06.621383 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:06.623875 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:06.623890 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:06.623905 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:06.624389 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:06.624887 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:06.624893 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:06.624898 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:06.625020 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:06.625944 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:06.625949 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:06.626081 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 17:51:06.626207 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 17:51:06.627306 (+ 1099us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":907,"spinlock_wait_cycles":3328}
W20260812 17:51:06.630215 4806 leader_election.cc:341] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785): Not found: Tablet not found: b92eecbdd8dd474a9e3ff61451347131
W20260812 17:51:06.630654 4809 leader_election.cc:341] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543): Not found: Tablet not found: b92eecbdd8dd474a9e3ff61451347131
I20260812 17:51:06.630748 4809 leader_election.cc:302] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dabf0cc6a8ab40049a42925dea4c2556; no voters: 05b77a06b3ba4aa8a85e826945a15c70, 8d84ffefdb3746e4a0971ebd6c98bd4c
I20260812 17:51:06.631081 5428 raft_consensus.cc:2726] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 17:51:06.777491 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:06.777017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:50377)
0812 17:51:06.777313 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:06.777377 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":112,"threads_started":1}
I20260812 17:51:07.398430 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:07.397902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:50013)
0812 17:51:07.398228 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:07.398304 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":98,"threads_started":1}
I20260812 17:51:07.568537 5439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:07.568083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:57455)
0812 17:51:07.568369 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:07.568436 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
I20260812 17:51:07.779019 5440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:07.778469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:52147)
0812 17:51:07.778816 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:07.778888 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":103,"threads_started":1}
I20260812 17:51:08.208585 5441 raft_consensus.cc:482] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:51:08.208863 5441 raft_consensus.cc:504] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:08.209407 5441 leader_election.cc:288] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785), 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543)
W20260812 17:51:08.210391 4809 leader_election.cc:341] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543): Not found: Tablet not found: b92eecbdd8dd474a9e3ff61451347131
W20260812 17:51:08.210542 4806 leader_election.cc:341] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785): Not found: Tablet not found: b92eecbdd8dd474a9e3ff61451347131
I20260812 17:51:08.210647 4806 leader_election.cc:302] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dabf0cc6a8ab40049a42925dea4c2556; no voters: 05b77a06b3ba4aa8a85e826945a15c70, 8d84ffefdb3746e4a0971ebd6c98bd4c
I20260812 17:51:08.210817 5441 raft_consensus.cc:2726] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 17:51:08.400022 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:08.399404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:41461)
0812 17:51:08.399802 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:08.399872 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":168,"threads_started":1}
I20260812 17:51:08.569782 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:08.569324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:37549)
0812 17:51:08.569623 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:08.569681 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":81,"threads_started":1}
I20260812 17:51:08.780470 5445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:08.780021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:45935)
0812 17:51:08.780304 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:08.780367 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":83,"threads_started":1}
I20260812 17:51:09.401597 5446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:09.401018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:56547)
0812 17:51:09.401386 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:09.401466 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":88,"threads_started":1}
I20260812 17:51:09.571134 5452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:09.570619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:42489)
0812 17:51:09.570953 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:09.571014 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":94,"threads_started":1}
I20260812 17:51:09.781865 5453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:09.781385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:39473)
0812 17:51:09.781705 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:09.781765 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":110,"threads_started":1}
I20260812 17:51:10.403477 5454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:10.402800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:57625)
0812 17:51:10.403254 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:10.403332 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":120,"threads_started":1}
I20260812 17:51:10.572646 5456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:10.572221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:36925)
0812 17:51:10.572481 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:10.572540 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":76,"threads_started":1}
I20260812 17:51:10.783310 5459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:10.782829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:40471)
0812 17:51:10.783129 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:10.783192 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":92,"threads_started":1}
I20260812 17:51:10.878937 5460 raft_consensus.cc:482] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:51:10.879084 5460 raft_consensus.cc:504] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:10.879762 5460 leader_election.cc:288] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785), 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543)
W20260812 17:51:10.880546 4809 leader_election.cc:341] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543): Not found: Tablet not found: b92eecbdd8dd474a9e3ff61451347131
W20260812 17:51:10.880715 4806 leader_election.cc:341] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785): Not found: Tablet not found: b92eecbdd8dd474a9e3ff61451347131
I20260812 17:51:10.880815 4806 leader_election.cc:302] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dabf0cc6a8ab40049a42925dea4c2556; no voters: 05b77a06b3ba4aa8a85e826945a15c70, 8d84ffefdb3746e4a0971ebd6c98bd4c
I20260812 17:51:10.880993 5460 raft_consensus.cc:2726] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 17:51:11.405247 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:11.404621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:52051)
0812 17:51:11.405001 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:11.405097 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":110,"threads_started":1}
I20260812 17:51:11.574807 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:11.574012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:32995)
0812 17:51:11.574473 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:11.574612 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":171,"threads_started":1}
I20260812 17:51:11.784763 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:11.784267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:51233)
0812 17:51:11.784597 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:11.784659 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":94,"threads_started":1}
I20260812 17:51:12.406658 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:12.406222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:38569)
0812 17:51:12.406495 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:12.406556 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":86,"threads_started":1}
I20260812 17:51:12.576603 5467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:12.576091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:39035)
0812 17:51:12.576417 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:12.576481 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":79,"threads_started":1}
I20260812 17:51:12.786268 5468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:12.785742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:34043)
0812 17:51:12.786083 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:12.786158 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":82,"threads_started":1}
I20260812 17:51:13.408241 5469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:13.407725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:60573)
0812 17:51:13.408044 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:13.408138 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":85,"threads_started":1}
I20260812 17:51:13.577982 5473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:13.577520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:52987)
0812 17:51:13.577813 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:13.577879 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":95,"threads_started":1}
I20260812 17:51:13.787775 5475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:13.787223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:36523)
0812 17:51:13.787603 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:13.787673 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":98,"threads_started":1}
I20260812 17:51:14.409731 5478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:14.409251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:58939)
0812 17:51:14.409559 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:14.409626 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":94,"threads_started":1}
I20260812 17:51:14.579365 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:14.578892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:46867)
0812 17:51:14.579202 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:14.579263 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":81,"threads_started":1}
I20260812 17:51:14.789211 5481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:14.788683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:55061)
0812 17:51:14.789009 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:14.789071 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":102,"threads_started":1}
I20260812 17:51:15.322520 5483 raft_consensus.cc:482] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:51:15.322691 5483 raft_consensus.cc:504] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:15.323200 5483 leader_election.cc:288] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785), 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543)
W20260812 17:51:15.323987 4809 leader_election.cc:341] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543): Not found: Tablet not found: b92eecbdd8dd474a9e3ff61451347131
W20260812 17:51:15.324132 4806 leader_election.cc:341] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785): Not found: Tablet not found: b92eecbdd8dd474a9e3ff61451347131
I20260812 17:51:15.324247 4806 leader_election.cc:302] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dabf0cc6a8ab40049a42925dea4c2556; no voters: 05b77a06b3ba4aa8a85e826945a15c70, 8d84ffefdb3746e4a0971ebd6c98bd4c
I20260812 17:51:15.324401 5483 raft_consensus.cc:2726] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 17:51:15.411290 5484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:15.410799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:58761)
0812 17:51:15.411121 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:15.411185 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":99,"threads_started":1}
I20260812 17:51:15.580914 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:15.580397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:50725)
0812 17:51:15.580740 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:15.580798 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":110,"threads_started":1}
I20260812 17:51:15.790727 5486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:15.790192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:52345)
0812 17:51:15.790522 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:15.790617 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":88,"threads_started":1}
I20260812 17:51:16.412675 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:16.412213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:45721)
0812 17:51:16.412499 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:16.412560 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":85,"threads_started":1}
I20260812 17:51:16.582357 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:16.581857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:48169)
0812 17:51:16.582184 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:16.582250 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":86,"threads_started":1}
I20260812 17:51:16.792315 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:16.791801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:41645)
0812 17:51:16.792134 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:16.792206 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":99,"threads_started":1}
I20260812 17:51:17.414227 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:17.413682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:47651)
0812 17:51:17.414032 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:17.414105 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":108,"threads_started":1}
I20260812 17:51:17.583814 5491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:17.583211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:35607)
0812 17:51:17.583497 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:17.583708 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":99,"threads_started":1}
I20260812 17:51:17.793830 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:17.793351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:53155)
0812 17:51:17.793667 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:17.793730 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":127,"threads_started":1}
I20260812 17:51:18.042340 5495 raft_consensus.cc:482] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:51:18.042500 5495 raft_consensus.cc:504] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b77a06b3ba4aa8a85e826945a15c70" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 40785 } } peers { permanent_uuid: "8d84ffefdb3746e4a0971ebd6c98bd4c" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 34543 } } peers { permanent_uuid: "dabf0cc6a8ab40049a42925dea4c2556" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37989 } }
I20260812 17:51:18.043015 5495 leader_election.cc:288] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785), 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543)
W20260812 17:51:18.044157 4806 leader_election.cc:341] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05b77a06b3ba4aa8a85e826945a15c70 (127.13.38.3:40785): Not found: Tablet not found: b92eecbdd8dd474a9e3ff61451347131
W20260812 17:51:18.044399 4809 leader_election.cc:341] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d84ffefdb3746e4a0971ebd6c98bd4c (127.13.38.2:34543): Not found: Tablet not found: b92eecbdd8dd474a9e3ff61451347131
I20260812 17:51:18.044510 4809 leader_election.cc:302] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dabf0cc6a8ab40049a42925dea4c2556; no voters: 05b77a06b3ba4aa8a85e826945a15c70, 8d84ffefdb3746e4a0971ebd6c98bd4c
I20260812 17:51:18.044679 5495 raft_consensus.cc:2726] T b92eecbdd8dd474a9e3ff61451347131 P dabf0cc6a8ab40049a42925dea4c2556 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 17:51:18.415788 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:18.415235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:40587)
0812 17:51:18.415573 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:18.415645 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":84,"threads_started":1}
I20260812 17:51:18.585392 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:18.584835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:36405)
0812 17:51:18.585208 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:18.585282 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":91,"threads_started":1}
I20260812 17:51:18.795292 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:18.794798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:53483)
0812 17:51:18.795118 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:18.795188 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":105,"threads_started":1}
I20260812 17:51:19.417354 5500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:19.416810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:35291)
0812 17:51:19.417152 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:19.417232 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":98,"threads_started":1}
I20260812 17:51:19.586884 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:19.586415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:46117)
0812 17:51:19.586703 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:19.586771 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":87,"threads_started":1}
I20260812 17:51:19.796763 5503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:19.796242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:59789)
0812 17:51:19.796580 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:19.796653 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":87,"threads_started":1}
I20260812 17:51:20.419050 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:20.418567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.1:56497)
0812 17:51:20.418876 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:20.418942 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":100,"threads_started":1}
I20260812 17:51:20.588587 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:20.588096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.2:33981)
0812 17:51:20.588410 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:20.588483 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":86,"threads_started":1}
I20260812 17:51:20.782752 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 4727
I20260812 17:51:20.798332 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:20.797863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:39111 (local address 127.13.38.3:42375)
0812 17:51:20.798162 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:20.798228 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":110,"threads_started":1}
I20260812 17:51:20.801842 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 4991
I20260812 17:51:20.819414 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 5214
2026-08-12T17:51:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20119 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:51:20 dist-test-slave-sb2z krb5kdc[5511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:51:20 dist-test-slave-sb2z krb5kdc[5511](info): set up 2 sockets
Aug 12 17:51:20 dist-test-slave-sb2z krb5kdc[5511](info): commencing operation
krb5kdc: starting...
W20260812 17:51:22.113710 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:51:22 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:51:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:51:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:51:22.258575 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34877
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:34877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:51:22.380005 5531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:22.380352 5531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:22.380442 5531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:22.384830 5531 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 17:51:22.384933 5531 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 17:51:22.384964 5531 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 17:51:22.385001 5531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:51:22.385040 5531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:22.385082 5531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:22.385118 5531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:51:22.385154 5531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:51:22.390728 5531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:34877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:22.391963 5531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:22.393040 5531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:22.402132 5545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:22.402287 5531 server_base.cc:1034] running on GCE node
W20260812 17:51:22.402153 5547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:22.402380 5552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:22.403101 5531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:22.404287 5531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:22.405480 5531 hybrid_clock.cc:648] HybridClock initialized: now 1786557082405465 us; error 29 us; skew 500 ppm
Aug 12 17:51:22 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557082, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:22.412460 5531 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:51:22.413509 5531 webserver.cc:460] Webserver started at http://127.13.38.62:40675/ using document root <none> and password file <none>
I20260812 17:51:22.414149 5531 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:22.414219 5531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:22.414461 5531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:51:22.416781 5531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "feab3fcd17b3406e96fdcc25b05be9cb"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "92061c8f9861fcea4dd4f24dc8f33b4d"
server_key_iv: "51094be76a4ccccdab21ad3241857029"
server_key_version: "encryptionkey@0"
I20260812 17:51:22.417409 5531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "feab3fcd17b3406e96fdcc25b05be9cb"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "92061c8f9861fcea4dd4f24dc8f33b4d"
server_key_iv: "51094be76a4ccccdab21ad3241857029"
server_key_version: "encryptionkey@0"
I20260812 17:51:22.421752 5531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:51:22.424630 5560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:22.426070 5531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:22.426214 5531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "feab3fcd17b3406e96fdcc25b05be9cb"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "92061c8f9861fcea4dd4f24dc8f33b4d"
server_key_iv: "51094be76a4ccccdab21ad3241857029"
server_key_version: "encryptionkey@0"
I20260812 17:51:22.426333 5531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:22.444487 5531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:22.448201 5531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:22.448451 5531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:22.456583 5531 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:34877
I20260812 17:51:22.457011 5642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:34877 every 8 connection(s)
I20260812 17:51:22.457691 5531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:51:22.460825 5645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:22.466841 5645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Bootstrap starting.
I20260812 17:51:22.467352 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 5531
I20260812 17:51:22.467497 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:51:22.467831 13464 external_mini_cluster.cc:1428] Setting key b82c36a5b24bd6c067fed867e2d91167
I20260812 17:51:22.469933 5645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:22.471096 5645 log.cc:824] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:22.473594 5645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: No bootstrap required, opened a new log
Aug 12 17:51:22 dist-test-slave-sb2z krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:22.478386 5645 raft_consensus.cc:348] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } }
I20260812 17:51:22.478657 5645 raft_consensus.cc:374] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:22.478726 5645 raft_consensus.cc:729] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feab3fcd17b3406e96fdcc25b05be9cb, State: Initialized, Role: FOLLOWER
I20260812 17:51:22.479308 5645 consensus_queue.cc:260] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } }
I20260812 17:51:22.479493 5645 raft_consensus.cc:388] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:51:22.479601 5645 raft_consensus.cc:482] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:51:22.479761 5645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:22.480821 5645 raft_consensus.cc:504] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } }
I20260812 17:51:22.481310 5645 leader_election.cc:302] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: feab3fcd17b3406e96fdcc25b05be9cb; no voters:
I20260812 17:51:22.481773 5645 leader_election.cc:288] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:51:22.481873 5650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:51:22.482106 5650 raft_consensus.cc:686] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 1 LEADER]: Becoming Leader. State: Replica: feab3fcd17b3406e96fdcc25b05be9cb, State: Running, Role: LEADER
I20260812 17:51:22.482584 5650 consensus_queue.cc:237] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } }
I20260812 17:51:22.483343 5645 sys_catalog.cc:564] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:51:22.484555 5648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:22.469409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33408 (local address 127.13.38.62:34877)
0812 17:51:22.469887 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:22.469905 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:22.469945 (+ 40us) server_negotiation.cc:407] Connection header received
0812 17:51:22.470697 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:22.470716 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:22.471075 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:22.471511 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:22.472438 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:22.473487 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:22.474500 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:22.475004 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:22.475190 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:22.478363 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:22.478368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:22.478385 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:22.478417 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:22.480480 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:22.480991 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:22.480999 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:22.481008 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:22.481176 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:22.481505 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:22.481509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:22.481511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:22.482055 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:22.482993 (+ 938us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:22.483151 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 17:51:22.483484 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":166,"threads_started":1}
I20260812 17:51:22.484565 5653 sys_catalog.cc:455] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } } }
I20260812 17:51:22.484815 5651 sys_catalog.cc:455] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader feab3fcd17b3406e96fdcc25b05be9cb. Latest consensus state: current_term: 1 leader_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } } }
I20260812 17:51:22.484956 5653 sys_catalog.cc:458] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: This master's current role is: LEADER
I20260812 17:51:22.484956 5651 sys_catalog.cc:458] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: This master's current role is: LEADER
I20260812 17:51:22.486377 5662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:51:22.490057 5662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:51:22.497033 5662 catalog_manager.cc:1347] Generated new cluster ID: c683fa90a27d4494ad348c14efcdb69a
I20260812 17:51:22.497222 5662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:51:22.509380 5662 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:51:22.510915 5662 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:51:22.518533 5662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Generated new TSK 0
I20260812 17:51:22.519474 5662 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:51:22.555094 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34877
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:51:22.683912 5681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:22.684223 5681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:22.684273 5681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:22.688943 5681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:22.689020 5681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:22.689087 5681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:51:22.694940 5681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:22.696188 5681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:22.697235 5681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:22.707600 5689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:22.707589 5694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:22.707593 5690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:22.708055 5681 server_base.cc:1034] running on GCE node
I20260812 17:51:22.708560 5681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:22.709295 5681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:22.710507 5681 hybrid_clock.cc:648] HybridClock initialized: now 1786557082710489 us; error 40 us; skew 500 ppm
Aug 12 17:51:22 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557082, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:22.713721 5681 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:51:22.714701 5681 webserver.cc:460] Webserver started at http://127.13.38.1:36377/ using document root <none> and password file <none>
I20260812 17:51:22.715337 5681 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:22.715442 5681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:22.715734 5681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:51:22.717968 5681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "f3118249b0f540efaefb0701970a08ff"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "90ca39a0d1267a41f7f10991c64f1df6"
server_key_iv: "5032297ad3e1b410ca786ef7d32b50ac"
server_key_version: "encryptionkey@0"
I20260812 17:51:22.718595 5681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "f3118249b0f540efaefb0701970a08ff"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "90ca39a0d1267a41f7f10991c64f1df6"
server_key_iv: "5032297ad3e1b410ca786ef7d32b50ac"
server_key_version: "encryptionkey@0"
I20260812 17:51:22.722708 5681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 17:51:22.725473 5702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:22.726857 5681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:51:22.727020 5681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "f3118249b0f540efaefb0701970a08ff"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "90ca39a0d1267a41f7f10991c64f1df6"
server_key_iv: "5032297ad3e1b410ca786ef7d32b50ac"
server_key_version: "encryptionkey@0"
I20260812 17:51:22.727167 5681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:22.750119 5681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:22.754026 5681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:22.754289 5681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:22.754994 5681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:22.755981 5681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:51:22.756080 5681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:22.756182 5681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:51:22.756237 5681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:22.766117 5681 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:34899
I20260812 17:51:22.766139 5885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:34899 every 8 connection(s)
I20260812 17:51:22.767135 5681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:51:22.774312 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 5681
I20260812 17:51:22.774475 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:51:22.774801 13464 external_mini_cluster.cc:1428] Setting key bae0138afb0c506bdddb23bbec6537dc
Aug 12 17:51:22 dist-test-slave-sb2z krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557082, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:22.783414 5648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:22.769203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:33865 (local address 127.13.38.62:34877)
0812 17:51:22.769398 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:22.769403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:22.770341 (+ 938us) server_negotiation.cc:407] Connection header received
0812 17:51:22.771844 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:22.771847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:22.771905 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:22.771999 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:22.773361 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:22.773930 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:22.775777 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:22.776057 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:22.776201 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:22.779275 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:22.779279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:22.779281 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:22.779308 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:22.780935 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:22.781533 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:22.781538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:22.781543 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:22.781654 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:22.782063 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:22.782067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:22.782068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:22.782283 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:22.782406 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:22.783092 (+ 686us) server_negotiation.cc:299] Negotiation successful
0812 17:51:22.783242 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 17:51:22.784420 5889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:22.769644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:33865)
0812 17:51:22.770161 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:22.770210 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:22.771482 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:22.772192 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:22.772200 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:22.772617 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:22.773136 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:22.773152 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:22.774102 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:22.774107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:22.775605 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:22.775619 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:22.776256 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:22.776261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:22.776468 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:22.777188 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:22.777206 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:22.779055 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:22.781087 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:22.781094 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:22.781113 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:22.781409 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:22.781782 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:22.781785 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:22.781787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:22.781948 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:22.782413 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:22.782424 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:22.782795 (+ 371us) client_negotiation.cc:769] Sending connection context
0812 17:51:22.783047 (+ 252us) client_negotiation.cc:240] Negotiation successful
0812 17:51:22.783424 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":131,"threads_started":1}
I20260812 17:51:22.785868 5887 heartbeater.cc:347] Connected to a master server at 127.13.38.62:34877
I20260812 17:51:22.786162 5887 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:51:22.786764 5887 heartbeater.cc:511] Master 127.13.38.62:34877 requested a full tablet report, sending...
I20260812 17:51:22.788443 5582 ts_manager.cc:194] Registered new tserver with Master: f3118249b0f540efaefb0701970a08ff (127.13.38.1:34899)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
I20260812 17:51:22.790113 5582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:33865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:51:22.811115 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34877
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:51:22.935792 5893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:22.936137 5893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:22.936239 5893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:22.940459 5893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:22.940579 5893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:22.940690 5893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:51:22.946426 5893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:22.947804 5893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:22.948949 5893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:22.959607 5906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:22.959631 5902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:22.959643 5904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:22.959973 5893 server_base.cc:1034] running on GCE node
I20260812 17:51:22.960542 5893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:22.961293 5893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:22.962628 5893 hybrid_clock.cc:648] HybridClock initialized: now 1786557082962595 us; error 70 us; skew 500 ppm
Aug 12 17:51:22 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557082, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:22.965687 5893 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:51:22.966645 5893 webserver.cc:460] Webserver started at http://127.13.38.2:35685/ using document root <none> and password file <none>
I20260812 17:51:22.967286 5893 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:22.967386 5893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:22.967724 5893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:51:22.970140 5893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "b49e090a6a244faab880eeb69dd73de6"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "797ad95b938e56c8c68be10b1d34f7a1"
server_key_iv: "b00968eccbba05543092b94317e62aeb"
server_key_version: "encryptionkey@0"
I20260812 17:51:22.970765 5893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "b49e090a6a244faab880eeb69dd73de6"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "797ad95b938e56c8c68be10b1d34f7a1"
server_key_iv: "b00968eccbba05543092b94317e62aeb"
server_key_version: "encryptionkey@0"
I20260812 17:51:22.974855 5893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:51:22.978056 5917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:22.980062 5893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:22.980223 5893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "b49e090a6a244faab880eeb69dd73de6"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "797ad95b938e56c8c68be10b1d34f7a1"
server_key_iv: "b00968eccbba05543092b94317e62aeb"
server_key_version: "encryptionkey@0"
I20260812 17:51:22.980371 5893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:22.996788 5893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:23.000344 5893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:23.000572 5893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:23.001266 5893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:23.002197 5893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:51:23.002249 5893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:23.002323 5893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:51:23.002364 5893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:23.012810 5893 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:39777
I20260812 17:51:23.012830 6087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:39777 every 8 connection(s)
I20260812 17:51:23.013900 5893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:51:23.020233 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 5893
I20260812 17:51:23.020422 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:51:23.020802 13464 external_mini_cluster.cc:1428] Setting key 5350f371b9a47ce2eca1cb21371edd8b
Aug 12 17:51:23 dist-test-slave-sb2z krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557082, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:23.031554 5648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.016218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:33897 (local address 127.13.38.62:34877)
0812 17:51:23.016364 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:23.016368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:23.017731 (+ 1363us) server_negotiation.cc:407] Connection header received
0812 17:51:23.018906 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:23.018910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:23.018975 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:23.019093 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:23.020583 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.021359 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.022717 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.023108 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.023238 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.026847 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:23.026851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:23.026853 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:23.026883 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:23.028850 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:23.029627 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:23.029631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:23.029635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:23.029723 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:23.030130 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:23.030134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:23.030136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:23.030400 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:23.030543 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:23.031210 (+ 667us) server_negotiation.cc:299] Negotiation successful
0812 17:51:23.031353 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 17:51:23.032580 6090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.016527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:33897)
0812 17:51:23.017527 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:23.017584 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:23.018671 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:23.019295 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:23.019302 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:23.019789 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:23.020336 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.020351 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.021536 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.021539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.022588 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.022599 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.023284 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.023286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.023442 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.024209 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:23.024228 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:23.026612 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:23.029012 (+ 2400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:23.029020 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:23.029043 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:23.029504 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:23.029852 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:23.029855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:23.029857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:23.030027 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:23.030549 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:23.030560 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:23.030854 (+ 294us) client_negotiation.cc:769] Sending connection context
0812 17:51:23.031163 (+ 309us) client_negotiation.cc:240] Negotiation successful
0812 17:51:23.031578 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":562,"threads_started":1}
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:51:23.034052 6088 heartbeater.cc:347] Connected to a master server at 127.13.38.62:34877
I20260812 17:51:23.034381 6088 heartbeater.cc:464] Registering TS with master...
I20260812 17:51:23.034977 6088 heartbeater.cc:511] Master 127.13.38.62:34877 requested a full tablet report, sending...
I20260812 17:51:23.036490 5582 ts_manager.cc:194] Registered new tserver with Master: b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
I20260812 17:51:23.037428 5582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:33897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:51:23.059300 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34877
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:51:23.181731 6096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:23.182116 6096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:23.182204 6096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:23.187500 6096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:23.187638 6096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:23.187747 6096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:51:23.193847 6096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:34877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:23.195127 6096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:23.196269 6096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:23.207671 6106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:23.207758 6105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:23.207664 6108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:23.207791 6096 server_base.cc:1034] running on GCE node
I20260812 17:51:23.208500 6096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:23.209273 6096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:23.210492 6096 hybrid_clock.cc:648] HybridClock initialized: now 1786557083210446 us; error 68 us; skew 500 ppm
Aug 12 17:51:23 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557083, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:23.213860 6096 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:51:23.214986 6096 webserver.cc:460] Webserver started at http://127.13.38.3:38647/ using document root <none> and password file <none>
I20260812 17:51:23.215727 6096 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:23.215834 6096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:23.216187 6096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:51:23.218640 6096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "7c7cf1bf258e42808c7808a7d9513838"
format_stamp: "Formatted at 2026-08-12 17:51:23 on dist-test-slave-sb2z"
server_key: "00996d975a4d29b0a41a55aa31af4edb"
server_key_iv: "7e3c43c7f2020f223c2595b650e105c2"
server_key_version: "encryptionkey@0"
I20260812 17:51:23.219359 6096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "7c7cf1bf258e42808c7808a7d9513838"
format_stamp: "Formatted at 2026-08-12 17:51:23 on dist-test-slave-sb2z"
server_key: "00996d975a4d29b0a41a55aa31af4edb"
server_key_iv: "7e3c43c7f2020f223c2595b650e105c2"
server_key_version: "encryptionkey@0"
I20260812 17:51:23.223887 6096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:51:23.226814 6118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:23.228489 6096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:23.228689 6096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "7c7cf1bf258e42808c7808a7d9513838"
format_stamp: "Formatted at 2026-08-12 17:51:23 on dist-test-slave-sb2z"
server_key: "00996d975a4d29b0a41a55aa31af4edb"
server_key_iv: "7e3c43c7f2020f223c2595b650e105c2"
server_key_version: "encryptionkey@0"
I20260812 17:51:23.228871 6096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:23.246799 6096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:23.250382 6096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:23.250629 6096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:23.251323 6096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:23.252351 6096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:51:23.252434 6096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:23.252518 6096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:51:23.252566 6096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:23.263279 6096 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:34015
I20260812 17:51:23.263309 6294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:34015 every 8 connection(s)
I20260812 17:51:23.264360 6096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:51:23.268381 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 6096
I20260812 17:51:23.268543 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:51:23.268905 13464 external_mini_cluster.cc:1428] Setting key 2ab347bd7067039a8e307f801b8564f1
Aug 12 17:51:23 dist-test-slave-sb2z krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557083, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:23.280768 5648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.266584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:47553 (local address 127.13.38.62:34877)
0812 17:51:23.266752 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:23.266756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:23.267814 (+ 1058us) server_negotiation.cc:407] Connection header received
0812 17:51:23.269603 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:23.269607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:23.269684 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:23.269800 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:23.271311 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.272026 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.273245 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.273482 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.273574 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.276440 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:23.276444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:23.276446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:23.276476 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:23.277965 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:23.278578 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:23.278597 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:23.278598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:23.278685 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:23.279335 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:23.279338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:23.279339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:23.279673 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:23.279772 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:23.280498 (+ 726us) server_negotiation.cc:299] Negotiation successful
0812 17:51:23.280627 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 17:51:23.281771 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.267043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:47553)
0812 17:51:23.267638 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:23.267690 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:23.269336 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:23.270024 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:23.270037 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:23.270582 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:23.271127 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.271143 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.272185 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.272188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.273107 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.273127 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.273660 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.273663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.273851 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.274614 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:23.274633 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:23.276249 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:23.278111 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:23.278118 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:23.278133 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:23.278466 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:23.278815 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:23.278818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:23.278820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:23.279054 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:23.279822 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:23.279829 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:23.280199 (+ 370us) client_negotiation.cc:769] Sending connection context
0812 17:51:23.280414 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 17:51:23.280738 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":141,"threads_started":1}
I20260812 17:51:23.283448 6295 heartbeater.cc:347] Connected to a master server at 127.13.38.62:34877
I20260812 17:51:23.283767 6295 heartbeater.cc:464] Registering TS with master...
I20260812 17:51:23.284425 6295 heartbeater.cc:511] Master 127.13.38.62:34877 requested a full tablet report, sending...
I20260812 17:51:23.285739 5581 ts_manager.cc:194] Registered new tserver with Master: 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015)
I20260812 17:51:23.286443 5581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:47553
I20260812 17:51:23.296404 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:51:23.307054 5648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.298661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33410 (local address 127.13.38.62:34877)
0812 17:51:23.298796 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:23.298800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:23.299123 (+ 323us) server_negotiation.cc:407] Connection header received
0812 17:51:23.299215 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:23.299217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:23.299264 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:23.299340 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:23.300244 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.300768 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.301689 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.301903 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.301987 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.303640 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:23.303643 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:23.303646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:23.303672 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:23.305122 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:23.305617 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:23.305622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:23.305626 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:23.305719 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:23.306037 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:23.306041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:23.306043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:23.306300 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:23.306463 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:23.306732 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 17:51:23.306866 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 17:51:23.310696 5581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:51:23.314227 5581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:51:23.332625 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.326111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:34899 (local address 127.0.0.1:34056)
0812 17:51:23.326889 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:23.326906 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:23.327002 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:23.327735 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:23.327741 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:23.327784 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:23.327950 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.327959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.329559 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.329562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.330769 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.330853 (+ 84us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.332010 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.332014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.332134 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.332189 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 17:51:23.332288 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 17:51:23.332416 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"spinlock_wait_cycles":2432,"thread_start_us":84,"threads_started":1}
I20260812 17:51:23.333024 6311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.326016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58532 (local address 127.13.38.2:39777)
0812 17:51:23.327264 (+ 1248us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:23.327276 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:23.327295 (+ 19us) server_negotiation.cc:407] Connection header received
0812 17:51:23.327360 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:23.327364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:23.327545 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:23.327728 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:23.328100 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.329149 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.331195 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.332125 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.332198 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.332320 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:23.332724 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 17:51:23.332848 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1144,"thread_start_us":129,"threads_started":1}
I20260812 17:51:23.333483 6316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.326898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49656 (local address 127.13.38.3:34015)
0812 17:51:23.328083 (+ 1185us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:23.328096 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:23.328130 (+ 34us) server_negotiation.cc:407] Connection header received
0812 17:51:23.328208 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:23.328213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:23.328415 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:23.328576 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:23.328793 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.329972 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.331606 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.332465 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.332572 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.332718 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:23.333157 (+ 439us) server_negotiation.cc:299] Negotiation successful
0812 17:51:23.333274 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1063,"thread_start_us":197,"threads_started":1}
I20260812 17:51:23.333016 6314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.326719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:34015 (local address 127.0.0.1:49656)
0812 17:51:23.327732 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:23.327746 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:23.327835 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:23.328581 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:23.328585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:23.328607 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:23.328689 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.328694 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.330129 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.330133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.331458 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.331469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.332594 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.332601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.332710 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.332734 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 17:51:23.332808 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 17:51:23.332875 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"thread_start_us":115,"threads_started":1}
I20260812 17:51:23.333747 6315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.326747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34056 (local address 127.13.38.1:34899)
0812 17:51:23.327181 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:23.327190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:23.327216 (+ 26us) server_negotiation.cc:407] Connection header received
0812 17:51:23.327300 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:23.327306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:23.327553 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:23.328123 (+ 570us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:23.328267 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.329419 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.331002 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.331840 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.333242 (+ 1402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.333410 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:23.333471 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 17:51:23.333565 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":122,"threads_started":1}
I20260812 17:51:23.333889 6310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.325868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:39777 (local address 127.0.0.1:58532)
0812 17:51:23.326298 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:23.326356 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:23.326498 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:23.327733 (+ 1235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:23.327741 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:23.327789 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:23.327965 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.327973 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.329354 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.329358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.331037 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.331049 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.332435 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.332438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.332508 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.332562 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 17:51:23.333258 (+ 696us) client_negotiation.cc:240] Negotiation successful
0812 17:51:23.333339 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":1,"spinlock_wait_cycles":4352,"thread_start_us":132,"threads_started":1}
I20260812 17:51:23.336864 5784 tablet_service.cc:1467] Processing CreateTablet for tablet d8b5709bad1a47f987dd6d69a478709b (DEFAULT_TABLE table=test-table [id=a4391b95639e446581b0b48f0b334cee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:51:23.337113 6189 tablet_service.cc:1467] Processing CreateTablet for tablet d8b5709bad1a47f987dd6d69a478709b (DEFAULT_TABLE table=test-table [id=a4391b95639e446581b0b48f0b334cee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:51:23.338013 5784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b5709bad1a47f987dd6d69a478709b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:23.338141 6189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b5709bad1a47f987dd6d69a478709b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:23.344960 6318 tablet_bootstrap.cc:492] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Bootstrap starting.
I20260812 17:51:23.346009 5988 tablet_service.cc:1467] Processing CreateTablet for tablet d8b5709bad1a47f987dd6d69a478709b (DEFAULT_TABLE table=test-table [id=a4391b95639e446581b0b48f0b334cee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:51:23.346472 6319 tablet_bootstrap.cc:492] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838: Bootstrap starting.
I20260812 17:51:23.347153 5988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b5709bad1a47f987dd6d69a478709b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:23.347658 6318 tablet_bootstrap.cc:654] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:23.348634 6318 log.cc:824] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:23.348883 6319 tablet_bootstrap.cc:654] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:23.350266 6319 log.cc:824] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:23.351229 6318 tablet_bootstrap.cc:492] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: No bootstrap required, opened a new log
I20260812 17:51:23.351454 6318 ts_tablet_manager.cc:1397] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260812 17:51:23.352528 6319 tablet_bootstrap.cc:492] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838: No bootstrap required, opened a new log
I20260812 17:51:23.352751 6319 ts_tablet_manager.cc:1397] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 17:51:23.355594 6322 tablet_bootstrap.cc:492] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6: Bootstrap starting.
I20260812 17:51:23.356388 6318 raft_consensus.cc:348] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:23.356701 6318 raft_consensus.cc:374] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:23.356758 6318 raft_consensus.cc:729] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3118249b0f540efaefb0701970a08ff, State: Initialized, Role: FOLLOWER
I20260812 17:51:23.356875 6319 raft_consensus.cc:348] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:23.357132 6319 raft_consensus.cc:374] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:23.357193 6319 raft_consensus.cc:729] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7cf1bf258e42808c7808a7d9513838, State: Initialized, Role: FOLLOWER
I20260812 17:51:23.357321 6318 consensus_queue.cc:260] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:23.357663 6319 consensus_queue.cc:260] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:23.358438 6295 heartbeater.cc:503] Master 127.13.38.62:34877 was elected leader, sending a full tablet report...
I20260812 17:51:23.358592 6319 ts_tablet_manager.cc:1428] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 17:51:23.358882 6322 tablet_bootstrap.cc:654] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:23.359655 6318 ts_tablet_manager.cc:1428] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260812 17:51:23.359884 6322 log.cc:824] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:23.360059 5887 heartbeater.cc:503] Master 127.13.38.62:34877 was elected leader, sending a full tablet report...
I20260812 17:51:23.362511 6322 tablet_bootstrap.cc:492] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6: No bootstrap required, opened a new log
I20260812 17:51:23.362752 6322 ts_tablet_manager.cc:1397] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 17:51:23.366816 6322 raft_consensus.cc:348] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:23.367146 6322 raft_consensus.cc:374] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:23.367216 6322 raft_consensus.cc:729] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b49e090a6a244faab880eeb69dd73de6, State: Initialized, Role: FOLLOWER
I20260812 17:51:23.367777 6322 consensus_queue.cc:260] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:23.368526 6088 heartbeater.cc:503] Master 127.13.38.62:34877 was elected leader, sending a full tablet report...
I20260812 17:51:23.368839 6322 ts_tablet_manager.cc:1428] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260812 17:51:23.515539 6089 tablet.cc:2369] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:51:23.515762 6296 tablet.cc:2369] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:51:23.518613 5888 tablet.cc:2369] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:51:23.696650 6324 raft_consensus.cc:482] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:51:23.697084 6324 raft_consensus.cc:504] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:23.698590 6324 leader_election.cc:288] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777), f3118249b0f540efaefb0701970a08ff (127.13.38.1:34899)
I20260812 17:51:23.702693 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.698823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:39777 (local address 127.13.38.3:33071)
0812 17:51:23.699035 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:23.699065 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:23.699173 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:23.699612 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:23.699615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:23.699639 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:23.699753 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.699759 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.700598 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.700601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.701408 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.701417 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.702247 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.702251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.702364 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.702383 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 17:51:23.702474 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 17:51:23.702548 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260812 17:51:23.702693 6311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.698966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:33071 (local address 127.13.38.2:39777)
0812 17:51:23.699125 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:23.699131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:23.699151 (+ 20us) server_negotiation.cc:407] Connection header received
0812 17:51:23.699307 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:23.699310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:23.699364 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:23.699483 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:23.699877 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.700462 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.701578 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.702115 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.702241 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.702307 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:23.702493 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 17:51:23.702563 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 17:51:23.703393 6333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.698949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:34899 (local address 127.13.38.3:37149)
0812 17:51:23.699371 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:23.699385 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:23.699489 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:23.700587 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:23.700589 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:23.700607 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:23.700710 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.700714 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.701435 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.701442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.702211 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:23.702219 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.702973 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.702979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:23.703070 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.703112 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 17:51:23.703192 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 17:51:23.703258 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":18,"thread_start_us":134,"threads_started":1}
I20260812 17:51:23.703397 6315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.699078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:37149 (local address 127.13.38.1:34899)
0812 17:51:23.699228 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:23.699235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:23.699489 (+ 254us) server_negotiation.cc:407] Connection header received
0812 17:51:23.699721 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:23.699724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:23.699767 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:23.699854 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:23.700833 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.701317 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.702341 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.702805 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.702921 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.702982 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:23.703212 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 17:51:23.703292 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 17:51:23.703424 6019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b5709bad1a47f987dd6d69a478709b" candidate_uuid: "7c7cf1bf258e42808c7808a7d9513838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b49e090a6a244faab880eeb69dd73de6" is_pre_election: true
I20260812 17:51:23.703840 6019 raft_consensus.cc:2454] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7cf1bf258e42808c7808a7d9513838 in term 0.
I20260812 17:51:23.704206 5817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b5709bad1a47f987dd6d69a478709b" candidate_uuid: "7c7cf1bf258e42808c7808a7d9513838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3118249b0f540efaefb0701970a08ff" is_pre_election: true
I20260812 17:51:23.704463 6123 leader_election.cc:302] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7cf1bf258e42808c7808a7d9513838, b49e090a6a244faab880eeb69dd73de6; no voters:
I20260812 17:51:23.704527 5817 raft_consensus.cc:2454] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7cf1bf258e42808c7808a7d9513838 in term 0.
I20260812 17:51:23.704792 6324 raft_consensus.cc:2781] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:51:23.704921 6324 raft_consensus.cc:482] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:51:23.704994 6324 raft_consensus.cc:3037] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:23.706218 6324 raft_consensus.cc:504] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:23.706614 6324 leader_election.cc:288] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [CANDIDATE]: Term 1 election: Requested vote from peers b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777), f3118249b0f540efaefb0701970a08ff (127.13.38.1:34899)
I20260812 17:51:23.707064 5817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b5709bad1a47f987dd6d69a478709b" candidate_uuid: "7c7cf1bf258e42808c7808a7d9513838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3118249b0f540efaefb0701970a08ff"
I20260812 17:51:23.707185 5817 raft_consensus.cc:3037] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:23.707752 6019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b5709bad1a47f987dd6d69a478709b" candidate_uuid: "7c7cf1bf258e42808c7808a7d9513838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b49e090a6a244faab880eeb69dd73de6"
I20260812 17:51:23.707924 6019 raft_consensus.cc:3037] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:23.708482 5817 raft_consensus.cc:2454] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7cf1bf258e42808c7808a7d9513838 in term 1.
I20260812 17:51:23.708856 6121 leader_election.cc:302] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7cf1bf258e42808c7808a7d9513838, f3118249b0f540efaefb0701970a08ff; no voters:
I20260812 17:51:23.709051 6324 raft_consensus.cc:2781] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:51:23.709317 6019 raft_consensus.cc:2454] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7cf1bf258e42808c7808a7d9513838 in term 1.
I20260812 17:51:23.709328 6324 raft_consensus.cc:686] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 1 LEADER]: Becoming Leader. State: Replica: 7c7cf1bf258e42808c7808a7d9513838, State: Running, Role: LEADER
I20260812 17:51:23.709905 6324 consensus_queue.cc:237] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:23.713402 5581 catalog_manager.cc:5495] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3). New cstate: current_term: 1 leader_uuid: "7c7cf1bf258e42808c7808a7d9513838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } health_report { overall_health: UNKNOWN } } }
I20260812 17:51:23.789011 6316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.784671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49668 (local address 127.13.38.3:34015)
0812 17:51:23.784877 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:23.784881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:23.784907 (+ 26us) server_negotiation.cc:407] Connection header received
0812 17:51:23.784997 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:23.784999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:23.785096 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:23.785266 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:51:23.785648 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.786277 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.787318 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:23.787591 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:23.787717 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:23.788093 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:51:23.788561 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:51:23.788710 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:23.788779 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 17:51:23.788845 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 17:51:23.795203 5817 raft_consensus.cc:1261] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Refusing update from remote peer 7c7cf1bf258e42808c7808a7d9513838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:51:23.795203 6019 raft_consensus.cc:1261] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6 [term 1 FOLLOWER]: Refusing update from remote peer 7c7cf1bf258e42808c7808a7d9513838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:51:23.796025 6334 consensus_queue.cc:1035] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:51:23.796243 6324 consensus_queue.cc:1035] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:51:23.805343 6338 mvcc.cc:205] Tried to move back new op lower bound from 7317737815217520640 to 7317737814878367744. Current Snapshot: MvccSnapshot[applied={T|T < 7317737815217520640}]
I20260812 17:51:23.805785 6341 mvcc.cc:205] Tried to move back new op lower bound from 7317737815217520640 to 7317737814878367744. Current Snapshot: MvccSnapshot[applied={T|T < 7317737815217520640}]
I20260812 17:51:23.808143 6340 mvcc.cc:205] Tried to move back new op lower bound from 7317737815217520640 to 7317737814878367744. Current Snapshot: MvccSnapshot[applied={T|T < 7317737815217520640}]
I20260812 17:51:23.814881 5580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:51:23.815109 5580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:51:23.818018 5580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Sending DeleteTablet for 3 replicas of tablet d8b5709bad1a47f987dd6d69a478709b
I20260812 17:51:23.819011 6189 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b5709bad1a47f987dd6d69a478709b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:23 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:49656
I20260812 17:51:23.818975 5784 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b5709bad1a47f987dd6d69a478709b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:23 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:34056
I20260812 17:51:23.818990 5988 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b5709bad1a47f987dd6d69a478709b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:23 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:58532
I20260812 17:51:23.819591 6362 tablet_replica.cc:330] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838: stopping tablet replica
I20260812 17:51:23.819787 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 5681
I20260812 17:51:23.842434 6362 raft_consensus.cc:2229] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:51:23.843083 6362 raft_consensus.cc:2258] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:51:23.845803 6362 ts_tablet_manager.cc:1905] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:51:23.819797 6357 tablet_replica.cc:330] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6: stopping tablet replica
I20260812 17:51:23.848857 6362 ts_tablet_manager.cc:1918] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:51:23.848992 6362 log.cc:1196] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/d8b5709bad1a47f987dd6d69a478709b
I20260812 17:51:23.849123 6357 raft_consensus.cc:2229] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:51:23.849423 6362 ts_tablet_manager.cc:1939] T d8b5709bad1a47f987dd6d69a478709b P 7c7cf1bf258e42808c7808a7d9513838: Deleting consensus metadata
I20260812 17:51:23.849692 6357 raft_consensus.cc:2258] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:51:23.851649 6357 ts_tablet_manager.cc:1905] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:51:23.852015 5564 catalog_manager.cc:4841] TS 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015): tablet d8b5709bad1a47f987dd6d69a478709b (table test-table [id=a4391b95639e446581b0b48f0b334cee]) successfully deleted
I20260812 17:51:23.854640 6357 ts_tablet_manager.cc:1918] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:51:23.854759 6357 log.cc:1196] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/d8b5709bad1a47f987dd6d69a478709b
I20260812 17:51:23.855101 6357 ts_tablet_manager.cc:1939] T d8b5709bad1a47f987dd6d69a478709b P b49e090a6a244faab880eeb69dd73de6: Deleting consensus metadata
W20260812 17:51:23.856725 5562 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:34899 (error 108)
I20260812 17:51:23.857007 5564 catalog_manager.cc:4841] TS b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777): tablet d8b5709bad1a47f987dd6d69a478709b (table test-table [id=a4391b95639e446581b0b48f0b334cee]) successfully deleted
I20260812 17:51:23.857069 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 5893
I20260812 17:51:23.864399 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:23.857785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:34899 (local address 127.0.0.1:34058)
0812 17:51:23.864179 (+ 6394us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:23.864272 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:34899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6293,"thread_start_us":1940,"threads_started":1}
W20260812 17:51:23.864636 5562 catalog_manager.cc:4584] TS f3118249b0f540efaefb0701970a08ff (127.13.38.1:34899): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8b5709bad1a47f987dd6d69a478709b: Network error: Client connection negotiation failed: client connection to 127.13.38.1:34899: connect: Connection refused (error 111)
I20260812 17:51:23.871984 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 6096
I20260812 17:51:23.888964 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 5531
I20260812 17:51:43.904757 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34877
--webserver_interface=127.13.38.62
--webserver_port=40675
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:34877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:51:44.031739 6366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:44.032050 6366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:44.032094 6366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:44.036337 6366 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 17:51:44.036418 6366 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 17:51:44.036435 6366 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 17:51:44.036450 6366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:51:44.036469 6366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:44.036489 6366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:44.036504 6366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:51:44.036520 6366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:51:44.042862 6366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:34877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=40675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:44.044274 6366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:44.045473 6366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:44.064177 6376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:44.064108 6377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:44.064237 6366 server_base.cc:1034] running on GCE node
W20260812 17:51:44.064119 6379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:44.065315 6366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:44.066754 6366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:44.067978 6366 hybrid_clock.cc:648] HybridClock initialized: now 1786557104067958 us; error 35 us; skew 500 ppm
Aug 12 17:51:44 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557104, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:44.079643 6366 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:51:44.081089 6366 webserver.cc:460] Webserver started at http://127.13.38.62:40675/ using document root <none> and password file <none>
I20260812 17:51:44.081802 6366 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:44.081936 6366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:44.086846 6366 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:51:44.089753 6388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:44.091324 6366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:51:44.091629 6366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "feab3fcd17b3406e96fdcc25b05be9cb"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "92061c8f9861fcea4dd4f24dc8f33b4d"
server_key_iv: "51094be76a4ccccdab21ad3241857029"
server_key_version: "encryptionkey@0"
I20260812 17:51:44.092403 6366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:44.111224 6366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:44.114989 6366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:44.115255 6366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:44.123803 6366 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:34877
I20260812 17:51:44.124325 6465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:34877 every 8 connection(s)
I20260812 17:51:44.125078 6366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:51:44.130190 6467 sys_catalog.cc:263] Verifying existing consensus state
I20260812 17:51:44.132285 6467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Bootstrap starting.
I20260812 17:51:44.135105 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 6366
I20260812 17:51:44.135640 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:34899
--local_ip_for_outbound_sockets=127.13.38.1
--tserver_master_addrs=127.13.38.62:34877
--webserver_port=36377
--webserver_interface=127.13.38.1
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:51:44.152889 6467 log.cc:824] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:44.164290 6467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 17:51:44.164803 6467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Bootstrap complete.
I20260812 17:51:44.169591 6467 raft_consensus.cc:348] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } }
I20260812 17:51:44.170090 6467 raft_consensus.cc:729] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: feab3fcd17b3406e96fdcc25b05be9cb, State: Initialized, Role: FOLLOWER
I20260812 17:51:44.170758 6467 consensus_queue.cc:260] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } }
I20260812 17:51:44.170898 6467 raft_consensus.cc:388] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:51:44.171006 6467 raft_consensus.cc:482] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:51:44.171128 6467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 1 FOLLOWER]: Advancing to term 2
I20260812 17:51:44.173517 6467 raft_consensus.cc:504] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } }
I20260812 17:51:44.173950 6467 leader_election.cc:302] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: feab3fcd17b3406e96fdcc25b05be9cb; no voters:
I20260812 17:51:44.174360 6467 leader_election.cc:288] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 17:51:44.174472 6471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 2 FOLLOWER]: Leader election won for term 2
I20260812 17:51:44.174708 6471 raft_consensus.cc:686] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 2 LEADER]: Becoming Leader. State: Replica: feab3fcd17b3406e96fdcc25b05be9cb, State: Running, Role: LEADER
I20260812 17:51:44.175165 6471 consensus_queue.cc:237] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } }
I20260812 17:51:44.175930 6467 sys_catalog.cc:564] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:51:44.176867 6475 sys_catalog.cc:455] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader feab3fcd17b3406e96fdcc25b05be9cb. Latest consensus state: current_term: 2 leader_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } } }
I20260812 17:51:44.177055 6475 sys_catalog.cc:458] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: This master's current role is: LEADER
I20260812 17:51:44.177523 6473 sys_catalog.cc:455] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } } }
I20260812 17:51:44.177630 6473 sys_catalog.cc:458] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: This master's current role is: LEADER
I20260812 17:51:44.182325 6503 catalog_manager.cc:1259] Loaded cluster ID: c683fa90a27d4494ad348c14efcdb69a
I20260812 17:51:44.182412 6503 catalog_manager.cc:1552] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: loading cluster ID for follower catalog manager: success
I20260812 17:51:44.185122 6503 catalog_manager.cc:1574] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: acquiring CA information for follower catalog manager: success
I20260812 17:51:44.187119 6503 catalog_manager.cc:1602] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 17:51:44.187570 6487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:51:44.190239 6487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:51:44.190752 6487 catalog_manager.cc:1259] Loaded cluster ID: c683fa90a27d4494ad348c14efcdb69a
I20260812 17:51:44.190819 6487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:51:44.191548 6487 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:51:44.192057 6487 catalog_manager.cc:5879] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Loaded TSK: 0
I20260812 17:51:44.214332 6487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Generated new TSK 1
I20260812 17:51:44.214658 6487 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 17:51:44.271843 6469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:44.272122 6469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:44.272169 6469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:44.276362 6469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:44.276432 6469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:44.276546 6469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:51:44.282378 6469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:34899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=36377
--tserver_master_addrs=127.13.38.62:34877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:44.283495 6469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:44.284641 6469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:44.295614 6519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:44.295560 6515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:44.295693 6516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:44.296092 6469 server_base.cc:1034] running on GCE node
I20260812 17:51:44.296581 6469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:44.297437 6469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:44.298820 6469 hybrid_clock.cc:648] HybridClock initialized: now 1786557104298766 us; error 77 us; skew 500 ppm
Aug 12 17:51:44 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557104, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:44.302033 6469 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:51:44.303158 6469 webserver.cc:460] Webserver started at http://127.13.38.1:36377/ using document root <none> and password file <none>
I20260812 17:51:44.303943 6469 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:44.304059 6469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:44.308481 6469 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 17:51:44.310909 6528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:44.312361 6469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:51:44.312511 6469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "f3118249b0f540efaefb0701970a08ff"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "90ca39a0d1267a41f7f10991c64f1df6"
server_key_iv: "5032297ad3e1b410ca786ef7d32b50ac"
server_key_version: "encryptionkey@0"
I20260812 17:51:44.313144 6469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:44.330112 6469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:44.333925 6469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:44.334167 6469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:44.334853 6469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:44.336113 6539 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 17:51:44.340777 6469 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 17:51:44.340973 6469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260812 17:51:44.341127 6469 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 17:51:44.343254 6469 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 17:51:44.343343 6469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 17:51:44.343710 6539 tablet_bootstrap.cc:492] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Bootstrap starting.
I20260812 17:51:44.356683 6469 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:34899
I20260812 17:51:44.356762 6699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:34899 every 8 connection(s)
I20260812 17:51:44.357940 6469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:51:44.364581 6539 log.cc:824] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:44.365131 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 6469
I20260812 17:51:44.365693 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:39777
--local_ip_for_outbound_sockets=127.13.38.2
--tserver_master_addrs=127.13.38.62:34877
--webserver_port=35685
--webserver_interface=127.13.38.2
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 17:51:44 dist-test-slave-sb2z krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557104, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:44.375180 6539 tablet_bootstrap.cc:492] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 17:51:44.375787 6539 tablet_bootstrap.cc:492] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Bootstrap complete.
I20260812 17:51:44.377642 6539 ts_tablet_manager.cc:1397] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Time spent bootstrapping tablet: real 0.034s user 0.024s sys 0.007s
I20260812 17:51:44.378881 6705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.360409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:55527)
0812 17:51:44.361132 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:44.361198 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:44.362483 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:44.363892 (+ 1409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:44.363901 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:44.364420 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:44.365137 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:44.365157 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.367017 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.367025 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:44.368366 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:44.368378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.369235 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.369240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:44.369463 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.370172 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:44.370192 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:44.372023 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:44.374563 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:44.374570 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:44.374586 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:44.374915 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:44.375447 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:44.375453 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:44.375455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:44.375727 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:44.376625 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:44.376631 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:44.377020 (+ 389us) client_negotiation.cc:769] Sending connection context
0812 17:51:44.377319 (+ 299us) client_negotiation.cc:240] Negotiation successful
0812 17:51:44.377762 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":185,"threads_started":1}
I20260812 17:51:44.378881 6702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.360439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:55527 (local address 127.13.38.62:34877)
0812 17:51:44.360947 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:44.360972 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:44.361355 (+ 383us) server_negotiation.cc:407] Connection header received
0812 17:51:44.362999 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:44.363016 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:44.363372 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:44.363788 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:44.365394 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.366805 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.368540 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.369058 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.369271 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.372278 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:44.372284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:44.372299 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:44.372332 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:44.374390 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:44.375092 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:44.375098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:44.375103 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:44.375283 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:44.375896 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:44.375900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:44.375906 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:44.376420 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:44.376537 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:44.377475 (+ 938us) server_negotiation.cc:299] Negotiation successful
0812 17:51:44.377927 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":160,"threads_started":1}
I20260812 17:51:44.381754 6539 raft_consensus.cc:348] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:44.382349 6539 raft_consensus.cc:729] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3118249b0f540efaefb0701970a08ff, State: Initialized, Role: FOLLOWER
I20260812 17:51:44.382493 6700 heartbeater.cc:347] Connected to a master server at 127.13.38.62:34877
I20260812 17:51:44.382774 6700 heartbeater.cc:464] Registering TS with master...
I20260812 17:51:44.382936 6539 consensus_queue.cc:260] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:44.383490 6700 heartbeater.cc:511] Master 127.13.38.62:34877 requested a full tablet report, sending...
I20260812 17:51:44.384080 6539 ts_tablet_manager.cc:1428] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260812 17:51:44.386459 6416 ts_manager.cc:194] Registered new tserver with Master: f3118249b0f540efaefb0701970a08ff (127.13.38.1:34899)
I20260812 17:51:44.390403 6416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:55527
I20260812 17:51:44.393405 6700 heartbeater.cc:503] Master 127.13.38.62:34877 was elected leader, sending a full tablet report...
Aug 12 17:51:44 dist-test-slave-sb2z krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557104, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for kudu/127.13.38.1@KRBTEST.COM
I20260812 17:51:44.403410 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.389869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41668 (local address 127.13.38.1:34899)
0812 17:51:44.390339 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:44.390353 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:44.390845 (+ 492us) server_negotiation.cc:407] Connection header received
0812 17:51:44.391053 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:44.391059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:44.391293 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:44.391490 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:44.392367 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.393483 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.394748 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.395839 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.395946 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.399103 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:44.399116 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:44.399129 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:44.399156 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:44.401136 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:44.401767 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:44.401774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:44.401775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:44.401863 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:44.402181 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:44.402184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:44.402186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:44.402497 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:44.402658 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:44.403049 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 17:51:44.403235 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":130,"threads_started":1}
I20260812 17:51:44.403410 6710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.389710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:34899 (local address 127.0.0.1:41668)
0812 17:51:44.390661 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:44.390719 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:44.390882 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:44.391490 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:44.391497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:44.392061 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:44.392201 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:44.392210 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.393647 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.393650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:44.394599 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:44.394613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.395985 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.395989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:44.396122 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.396904 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:44.396921 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:44.398891 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:44.401308 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:44.401321 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:44.401327 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:44.401621 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:44.401970 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:44.401976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:44.401980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:44.402079 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:44.402626 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:44.402633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:44.402784 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 17:51:44.402935 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 17:51:44.403204 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":125,"threads_started":1}
I20260812 17:51:44.404600 6599 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b5709bad1a47f987dd6d69a478709b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:23 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:41668
I20260812 17:51:44.405577 6715 tablet_replica.cc:330] stopping tablet replica
I20260812 17:51:44.405869 6715 raft_consensus.cc:2229] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:51:44.406098 6715 raft_consensus.cc:2258] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:51:44.407101 6715 ts_tablet_manager.cc:1905] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:51:44.411078 6715 ts_tablet_manager.cc:1918] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:51:44.411188 6715 log.cc:1196] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal/wals/d8b5709bad1a47f987dd6d69a478709b
I20260812 17:51:44.411482 6715 ts_tablet_manager.cc:1939] T d8b5709bad1a47f987dd6d69a478709b P f3118249b0f540efaefb0701970a08ff: Deleting consensus metadata
I20260812 17:51:44.413424 6389 catalog_manager.cc:4841] TS f3118249b0f540efaefb0701970a08ff (127.13.38.1:34899): tablet d8b5709bad1a47f987dd6d69a478709b (table test-table [id=a4391b95639e446581b0b48f0b334cee]) successfully deleted
W20260812 17:51:44.498631 6707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:44.498992 6707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:44.499080 6707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:44.503720 6707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:44.503865 6707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:44.503973 6707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:51:44.510023 6707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:39777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=35685
--tserver_master_addrs=127.13.38.62:34877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:44.511330 6707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:44.512784 6707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:44.524269 6723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:44.524312 6720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:44.524359 6707 server_base.cc:1034] running on GCE node
W20260812 17:51:44.524307 6726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:44.525168 6707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:44.525996 6707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:44.527233 6707 hybrid_clock.cc:648] HybridClock initialized: now 1786557104527206 us; error 48 us; skew 500 ppm
Aug 12 17:51:44 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557104, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:44.530604 6707 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:51:44.531769 6707 webserver.cc:460] Webserver started at http://127.13.38.2:35685/ using document root <none> and password file <none>
I20260812 17:51:44.532441 6707 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:44.532577 6707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:44.537433 6707 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 17:51:44.540380 6739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:44.542062 6707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:44.542248 6707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "b49e090a6a244faab880eeb69dd73de6"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "797ad95b938e56c8c68be10b1d34f7a1"
server_key_iv: "b00968eccbba05543092b94317e62aeb"
server_key_version: "encryptionkey@0"
I20260812 17:51:44.542920 6707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:44.561053 6707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:44.564872 6707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:44.565128 6707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:44.565814 6707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:44.566741 6707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:51:44.566816 6707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:44.566891 6707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:51:44.566944 6707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:44.577435 6707 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:39777
I20260812 17:51:44.577453 6902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:39777 every 8 connection(s)
I20260812 17:51:44.578636 6707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:51:44.586484 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 6707
I20260812 17:51:44.587011 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:34015
--local_ip_for_outbound_sockets=127.13.38.3
--tserver_master_addrs=127.13.38.62:34877
--webserver_port=38647
--webserver_interface=127.13.38.3
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 17:51:44 dist-test-slave-sb2z krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557104, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:44.596618 6702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.580853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:46301 (local address 127.13.38.62:34877)
0812 17:51:44.581051 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:44.581056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:44.582174 (+ 1118us) server_negotiation.cc:407] Connection header received
0812 17:51:44.583630 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:44.583638 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:44.583694 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:44.583780 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:44.585134 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.585699 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.587624 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.587876 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.587971 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.591736 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:44.591744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:44.591749 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:44.591796 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:44.593745 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:44.594403 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:44.594410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:44.594414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:44.594536 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:44.594918 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:44.594924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:44.594927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:44.595236 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:44.595360 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:44.596103 (+ 743us) server_negotiation.cc:299] Negotiation successful
0812 17:51:44.596298 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 17:51:44.597553 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.581355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:46301)
0812 17:51:44.581964 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:44.582011 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:44.583328 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:44.583982 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:44.583990 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:44.584381 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:44.584916 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:44.584929 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.585854 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.585858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:44.587366 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:44.587382 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.588062 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.588069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:44.588319 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.589277 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:44.589297 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:44.591445 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:44.593907 (+ 2462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:44.593914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:44.593931 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:44.594275 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:44.594644 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:44.594646 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:44.594654 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:44.594811 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:44.595355 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:44.595361 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:44.595731 (+ 370us) client_negotiation.cc:769] Sending connection context
0812 17:51:44.595948 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 17:51:44.596355 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":154,"threads_started":1}
I20260812 17:51:44.598973 6903 heartbeater.cc:347] Connected to a master server at 127.13.38.62:34877
I20260812 17:51:44.599303 6903 heartbeater.cc:464] Registering TS with master...
I20260812 17:51:44.600042 6903 heartbeater.cc:511] Master 127.13.38.62:34877 requested a full tablet report, sending...
I20260812 17:51:44.601305 6416 ts_manager.cc:194] Registered new tserver with Master: b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777)
I20260812 17:51:44.602170 6416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:46301
W20260812 17:51:44.717890 6908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:51:44.718255 6908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:51:44.718341 6908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:51:44.722869 6908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:51:44.722959 6908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:51:44.723027 6908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:51:44.728876 6908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:34015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=38647
--tserver_master_addrs=127.13.38.62:34877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:51:44.730093 6908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:51:44.731262 6908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:51:44.743553 6921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:44.743590 6918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:51:44.743582 6917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:51:44.743907 6908 server_base.cc:1034] running on GCE node
I20260812 17:51:44.744453 6908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:51:44.745203 6908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:51:44.746410 6908 hybrid_clock.cc:648] HybridClock initialized: now 1786557104746360 us; error 59 us; skew 500 ppm
Aug 12 17:51:44 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557104, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:51:44.749934 6908 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:51:44.751068 6908 webserver.cc:460] Webserver started at http://127.13.38.3:38647/ using document root <none> and password file <none>
I20260812 17:51:44.751828 6908 fs_manager.cc:359] Metadata directory not provided
I20260812 17:51:44.751922 6908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:51:44.756623 6908 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:51:44.759145 6930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:44.760716 6908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:51:44.760921 6908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "7c7cf1bf258e42808c7808a7d9513838"
format_stamp: "Formatted at 2026-08-12 17:51:23 on dist-test-slave-sb2z"
server_key: "00996d975a4d29b0a41a55aa31af4edb"
server_key_iv: "7e3c43c7f2020f223c2595b650e105c2"
server_key_version: "encryptionkey@0"
I20260812 17:51:44.761593 6908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:51:44.788487 6908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:51:44.792773 6908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:51:44.793061 6908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:51:44.793818 6908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:51:44.794866 6908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:51:44.794955 6908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:44.795045 6908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:51:44.795109 6908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:51:44.808687 6908 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:34015
I20260812 17:51:44.808705 7100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:34015 every 8 connection(s)
I20260812 17:51:44.810009 6908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:51:44.817930 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 6908
Aug 12 17:51:44 dist-test-slave-sb2z krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557104, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:51:44.827800 6702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.812402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:38689 (local address 127.13.38.62:34877)
0812 17:51:44.812609 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:44.812613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:44.813666 (+ 1053us) server_negotiation.cc:407] Connection header received
0812 17:51:44.815156 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:44.815159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:44.815226 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:44.815313 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:44.816984 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.817623 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.819395 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.819840 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.820015 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.822910 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:44.822915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:44.822918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:44.822949 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:44.825159 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:44.825961 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:44.825965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:44.825966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:44.826079 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:44.826520 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:44.826524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:44.826526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:44.826749 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:44.826869 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:44.827461 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 17:51:44.827626 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 17:51:44.828778 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.812872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:38689)
0812 17:51:44.813461 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:44.813523 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:44.814885 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:44.815682 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:44.815691 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:44.816127 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:51:44.816802 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:44.816820 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.817798 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.817801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:44.819211 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:44.819225 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.820035 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.820038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:44.820203 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.820835 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:51:44.820858 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:51:44.822712 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:51:44.825306 (+ 2594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:44.825317 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:44.825344 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:44.825830 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:44.826209 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:51:44.826212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:51:44.826214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:51:44.826403 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:51:44.826870 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:51:44.826879 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:51:44.827193 (+ 314us) client_negotiation.cc:769] Sending connection context
0812 17:51:44.827417 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 17:51:44.827853 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":127,"threads_started":1}
I20260812 17:51:44.829411 7106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.818793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39792 (local address 127.13.38.62:34877)
0812 17:51:44.819131 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:44.819136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:44.819153 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:51:44.819320 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:44.819326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:44.819387 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:44.819478 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:44.820559 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.821345 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.822351 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.822706 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.822832 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.825123 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:44.825133 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:44.825135 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:44.825164 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:44.826962 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:44.827487 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:44.827492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:44.827494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:44.827624 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:44.828013 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:44.828019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:44.828022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:44.828260 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:44.828519 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:44.828677 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 17:51:44.828808 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":128,"threads_started":1}
I20260812 17:51:44.830703 7101 heartbeater.cc:347] Connected to a master server at 127.13.38.62:34877
I20260812 17:51:44.831054 7101 heartbeater.cc:464] Registering TS with master...
I20260812 17:51:44.831872 7101 heartbeater.cc:511] Master 127.13.38.62:34877 requested a full tablet report, sending...
I20260812 17:51:44.833220 6413 ts_manager.cc:194] Registered new tserver with Master: 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015)
I20260812 17:51:44.834060 6413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:38689
I20260812 17:51:44.844746 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:51:44.849769 7106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.845536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39798 (local address 127.13.38.62:34877)
0812 17:51:44.845696 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:44.845700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:44.845852 (+ 152us) server_negotiation.cc:407] Connection header received
0812 17:51:44.845974 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:44.845977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:44.846141 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:44.846240 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:51:44.846547 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.847191 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.848331 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.848579 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.848674 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.849133 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:51:44.849310 (+ 177us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 17:51:44.849577 (+ 267us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39798: token expired
Metrics: {"server-negotiator.queue_time_us":70}
W20260812 17:51:44.849882 7106 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39798: token expired
W20260812 17:51:44.850330 6302 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.38.62:34877)
I20260812 17:51:44.858780 7106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.850817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.13.38.62:34877)
0812 17:51:44.850954 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:44.850957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:44.851003 (+ 46us) server_negotiation.cc:407] Connection header received
0812 17:51:44.851147 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:44.851150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:44.851192 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:44.851261 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:51:44.852071 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.852573 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.853455 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.853675 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.853812 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.855262 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:51:44.855266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:51:44.855271 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:51:44.855303 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:51:44.856893 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:44.857481 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:44.857485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:44.857487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:44.857576 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:51:44.857868 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:51:44.857872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:51:44.857873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:51:44.858064 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:51:44.858214 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:44.858474 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 17:51:44.858604 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:51:44.861764 6413 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:51:44.865368 6413 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:51:44.875378 6599 tablet_service.cc:1467] Processing CreateTablet for tablet 4ea821da2ae748859a01a17b2ad2a9f3 (DEFAULT_TABLE table=test-table [id=bf9b8eb3afb44524b2ed22557b954125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:51:44.876283 6599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ea821da2ae748859a01a17b2ad2a9f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:44.879928 6539 tablet_bootstrap.cc:492] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff: Bootstrap starting.
I20260812 17:51:44.880991 6710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.874275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:39777 (local address 127.0.0.1:51868)
0812 17:51:44.874440 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:44.874460 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:44.874566 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:44.875655 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:44.875660 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:44.875683 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:44.875827 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:44.875835 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.877642 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.877645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:44.878920 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:44.878933 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.880432 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.880436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:44.880587 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.880660 (+ 73us) client_negotiation.cc:769] Sending connection context
0812 17:51:44.880721 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 17:51:44.880820 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 17:51:44.881162 6539 tablet_bootstrap.cc:654] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:44.881775 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.874461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51868 (local address 127.13.38.2:39777)
0812 17:51:44.875075 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:44.875092 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:44.875132 (+ 40us) server_negotiation.cc:407] Connection header received
0812 17:51:44.875217 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:44.875222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:44.875411 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:44.876032 (+ 621us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:44.876236 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.877500 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.879108 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.880271 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.881233 (+ 962us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.881411 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:44.881482 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 17:51:44.881597 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":210,"threads_started":1}
I20260812 17:51:44.881947 7114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.874574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:34015 (local address 127.0.0.1:47766)
0812 17:51:44.874917 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:44.874935 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:44.875046 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:44.876813 (+ 1767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:44.876816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:44.876835 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:44.877106 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:44.877118 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.878413 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.878420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:44.879409 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:44.879420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.881458 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.881462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:44.881577 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.881597 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 17:51:44.881697 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 17:51:44.881780 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":2,"thread_start_us":111,"threads_started":1}
I20260812 17:51:44.883220 6539 tablet_bootstrap.cc:492] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff: No bootstrap required, opened a new log
I20260812 17:51:44.883360 6539 ts_tablet_manager.cc:1397] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 17:51:44.883436 7115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:44.874918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47766 (local address 127.13.38.3:34015)
0812 17:51:44.875377 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:44.875391 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:44.875419 (+ 28us) server_negotiation.cc:407] Connection header received
0812 17:51:44.875511 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:44.875516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:44.876285 (+ 769us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:44.876594 (+ 309us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:44.877305 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.878285 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.879588 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:44.881075 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:44.881201 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:44.881389 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:44.881738 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 17:51:44.882948 (+ 1210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":136,"threads_started":1}
I20260812 17:51:44.883903 6539 raft_consensus.cc:348] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:44.884056 6539 raft_consensus.cc:374] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:44.884114 6539 raft_consensus.cc:729] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3118249b0f540efaefb0701970a08ff, State: Initialized, Role: FOLLOWER
I20260812 17:51:44.884248 6539 consensus_queue.cc:260] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:44.884630 6539 ts_tablet_manager.cc:1428] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 17:51:44.887456 6813 tablet_service.cc:1467] Processing CreateTablet for tablet 4ea821da2ae748859a01a17b2ad2a9f3 (DEFAULT_TABLE table=test-table [id=bf9b8eb3afb44524b2ed22557b954125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:51:44.887770 7005 tablet_service.cc:1467] Processing CreateTablet for tablet 4ea821da2ae748859a01a17b2ad2a9f3 (DEFAULT_TABLE table=test-table [id=bf9b8eb3afb44524b2ed22557b954125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:51:44.888938 7005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ea821da2ae748859a01a17b2ad2a9f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:44.889212 6813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ea821da2ae748859a01a17b2ad2a9f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:51:44.897115 7121 tablet_bootstrap.cc:492] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838: Bootstrap starting.
I20260812 17:51:44.897153 7120 tablet_bootstrap.cc:492] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6: Bootstrap starting.
I20260812 17:51:44.899732 7121 tablet_bootstrap.cc:654] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:44.900210 7120 tablet_bootstrap.cc:654] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6: Neither blocks nor log segments found. Creating new log.
I20260812 17:51:44.901237 7121 log.cc:824] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:44.901239 7120 log.cc:824] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6: Log is configured to *not* fsync() on all Append() calls
I20260812 17:51:44.903482 7120 tablet_bootstrap.cc:492] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6: No bootstrap required, opened a new log
I20260812 17:51:44.903493 7121 tablet_bootstrap.cc:492] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838: No bootstrap required, opened a new log
I20260812 17:51:44.903790 7121 ts_tablet_manager.cc:1397] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260812 17:51:44.903785 7120 ts_tablet_manager.cc:1397] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260812 17:51:44.906891 7121 raft_consensus.cc:348] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:44.906890 7120 raft_consensus.cc:348] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:44.907205 7121 raft_consensus.cc:374] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:44.907195 7120 raft_consensus.cc:374] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:51:44.907274 7120 raft_consensus.cc:729] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b49e090a6a244faab880eeb69dd73de6, State: Initialized, Role: FOLLOWER
I20260812 17:51:44.907276 7121 raft_consensus.cc:729] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7cf1bf258e42808c7808a7d9513838, State: Initialized, Role: FOLLOWER
I20260812 17:51:44.907804 7120 consensus_queue.cc:260] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:44.907814 7121 consensus_queue.cc:260] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:44.908607 6903 heartbeater.cc:503] Master 127.13.38.62:34877 was elected leader, sending a full tablet report...
I20260812 17:51:44.909010 7120 ts_tablet_manager.cc:1428] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260812 17:51:44.909004 7121 ts_tablet_manager.cc:1428] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 17:51:44.909205 7101 heartbeater.cc:503] Master 127.13.38.62:34877 was elected leader, sending a full tablet report...
W20260812 17:51:45.061347 7102 tablet.cc:2369] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:51:45.080015 6904 tablet.cc:2369] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:51:45.109700 6701 tablet.cc:2369] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:51:45.209589 7117 raft_consensus.cc:482] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:51:45.209918 7117 raft_consensus.cc:504] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:45.211134 7117 leader_election.cc:288] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777), 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015)
I20260812 17:51:45.215790 7115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:45.212111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:55377 (local address 127.13.38.3:34015)
0812 17:51:45.212242 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:45.212248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:45.212328 (+ 80us) server_negotiation.cc:407] Connection header received
0812 17:51:45.212511 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:45.212514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:45.212604 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:45.212689 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:45.212892 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:45.213598 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:45.214708 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:45.215132 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:45.215256 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:45.215322 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:45.215567 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 17:51:45.215643 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 17:51:45.215816 7131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:45.211965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:34015 (local address 127.13.38.1:55377)
0812 17:51:45.212242 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:45.212257 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:45.212382 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:45.212709 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:45.212711 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:45.212726 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:45.212791 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:45.212795 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:45.213720 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:45.213723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:45.214578 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:45.214588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:45.215265 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:45.215269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:45.215357 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:45.215423 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 17:51:45.215518 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 17:51:45.215638 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":32,"thread_start_us":80,"threads_started":1}
I20260812 17:51:45.215808 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:45.211622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:35201 (local address 127.13.38.2:39777)
0812 17:51:45.211768 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:45.211772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:45.211919 (+ 147us) server_negotiation.cc:407] Connection header received
0812 17:51:45.212173 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:45.212178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:45.212238 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:45.212282 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:51:45.212645 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:45.213502 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:45.214734 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:45.215155 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:45.215282 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:45.215349 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:45.215594 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 17:51:45.215677 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 17:51:45.215823 7129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:45.211469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:39777 (local address 127.13.38.1:35201)
0812 17:51:45.211804 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:45.211821 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:51:45.211965 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:51:45.212418 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:51:45.212420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:51:45.212437 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:51:45.212531 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:45.212535 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:45.213643 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:45.213646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:45.214577 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:51:45.214589 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:51:45.215289 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:51:45.215292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:51:45.215384 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:45.215420 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 17:51:45.215518 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 17:51:45.215670 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":69,"thread_start_us":170,"threads_started":1}
I20260812 17:51:45.216719 6837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ea821da2ae748859a01a17b2ad2a9f3" candidate_uuid: "f3118249b0f540efaefb0701970a08ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b49e090a6a244faab880eeb69dd73de6" is_pre_election: true
I20260812 17:51:45.216737 7035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ea821da2ae748859a01a17b2ad2a9f3" candidate_uuid: "f3118249b0f540efaefb0701970a08ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7cf1bf258e42808c7808a7d9513838" is_pre_election: true
I20260812 17:51:45.217092 7035 raft_consensus.cc:2454] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3118249b0f540efaefb0701970a08ff in term 0.
I20260812 17:51:45.217101 6837 raft_consensus.cc:2454] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3118249b0f540efaefb0701970a08ff in term 0.
I20260812 17:51:45.218281 6535 leader_election.cc:302] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7cf1bf258e42808c7808a7d9513838, f3118249b0f540efaefb0701970a08ff; no voters:
I20260812 17:51:45.218633 7117 raft_consensus.cc:2781] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:51:45.218746 7117 raft_consensus.cc:482] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:51:45.218811 7117 raft_consensus.cc:3037] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:45.219974 7117 raft_consensus.cc:504] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:45.220379 7117 leader_election.cc:288] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 1 election: Requested vote from peers b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777), 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015)
I20260812 17:51:45.220935 6837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ea821da2ae748859a01a17b2ad2a9f3" candidate_uuid: "f3118249b0f540efaefb0701970a08ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b49e090a6a244faab880eeb69dd73de6"
I20260812 17:51:45.220981 7035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ea821da2ae748859a01a17b2ad2a9f3" candidate_uuid: "f3118249b0f540efaefb0701970a08ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c7cf1bf258e42808c7808a7d9513838"
I20260812 17:51:45.221066 6837 raft_consensus.cc:3037] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:45.221180 7035 raft_consensus.cc:3037] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:51:45.222297 7035 raft_consensus.cc:2454] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3118249b0f540efaefb0701970a08ff in term 1.
I20260812 17:51:45.222295 6837 raft_consensus.cc:2454] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3118249b0f540efaefb0701970a08ff in term 1.
I20260812 17:51:45.222716 6535 leader_election.cc:302] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7cf1bf258e42808c7808a7d9513838, f3118249b0f540efaefb0701970a08ff; no voters:
I20260812 17:51:45.222939 7117 raft_consensus.cc:2781] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:51:45.223161 7117 raft_consensus.cc:686] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 LEADER]: Becoming Leader. State: Replica: f3118249b0f540efaefb0701970a08ff, State: Running, Role: LEADER
I20260812 17:51:45.223459 7117 consensus_queue.cc:237] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:51:45.226763 6416 catalog_manager.cc:5495] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff reported cstate change: term changed from 0 to 1, leader changed from <none> to f3118249b0f540efaefb0701970a08ff (127.13.38.1). New cstate: current_term: 1 leader_uuid: "f3118249b0f540efaefb0701970a08ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } health_report { overall_health: HEALTHY } } }
I20260812 17:51:45.254798 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:45.250597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41676 (local address 127.13.38.1:34899)
0812 17:51:45.250938 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0812 17:51:45.250945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:51:45.250960 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:51:45.251093 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:51:45.251095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:51:45.251176 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:51:45.251350 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:51:45.251702 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:45.252272 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:45.253169 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:51:45.253412 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:51:45.253533 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:51:45.253834 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:51:45.254320 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:51:45.254462 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 17:51:45.254550 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 17:51:45.254611 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":132,"threads_started":1}
I20260812 17:51:45.259318 7035 raft_consensus.cc:1261] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838 [term 1 FOLLOWER]: Refusing update from remote peer f3118249b0f540efaefb0701970a08ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:51:45.259318 6837 raft_consensus.cc:1261] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6 [term 1 FOLLOWER]: Refusing update from remote peer f3118249b0f540efaefb0701970a08ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:51:45.260329 7117 consensus_queue.cc:1035] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:51:45.260613 7133 consensus_queue.cc:1035] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:51:45.268764 7138 mvcc.cc:205] Tried to move back new op lower bound from 7317737903134543872 to 7317737902998323200. Current Snapshot: MvccSnapshot[applied={T|T < 7317737903134543872}]
I20260812 17:51:45.271198 7140 mvcc.cc:205] Tried to move back new op lower bound from 7317737903134543872 to 7317737902998323200. Current Snapshot: MvccSnapshot[applied={T|T < 7317737903134543872}]
I20260812 17:51:45.278163 6416 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:51:45.278414 6416 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:51:45.281031 6416 catalog_manager.cc:5782] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Sending DeleteTablet for 3 replicas of tablet 4ea821da2ae748859a01a17b2ad2a9f3
I20260812 17:51:45.281996 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 6469
I20260812 17:51:45.282014 7005 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ea821da2ae748859a01a17b2ad2a9f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:45 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:47766
I20260812 17:51:45.282004 6813 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ea821da2ae748859a01a17b2ad2a9f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:51:45 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:51868
I20260812 17:51:45.286597 7159 tablet_replica.cc:330] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6: stopping tablet replica
I20260812 17:51:45.286937 7159 raft_consensus.cc:2229] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:51:45.287245 7159 raft_consensus.cc:2258] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:51:45.287706 7158 tablet_replica.cc:330] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838: stopping tablet replica
I20260812 17:51:45.288040 7158 raft_consensus.cc:2229] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:51:45.288355 7158 raft_consensus.cc:2258] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:51:45.290030 7159 ts_tablet_manager.cc:1905] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:51:45.290921 7158 ts_tablet_manager.cc:1905] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:51:45.293707 7159 ts_tablet_manager.cc:1918] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:51:45.293850 7159 log.cc:1196] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/4ea821da2ae748859a01a17b2ad2a9f3
I20260812 17:51:45.294087 7158 ts_tablet_manager.cc:1918] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:51:45.294205 7158 log.cc:1196] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/4ea821da2ae748859a01a17b2ad2a9f3
I20260812 17:51:45.294256 7159 ts_tablet_manager.cc:1939] T 4ea821da2ae748859a01a17b2ad2a9f3 P b49e090a6a244faab880eeb69dd73de6: Deleting consensus metadata
I20260812 17:51:45.294543 7158 ts_tablet_manager.cc:1939] T 4ea821da2ae748859a01a17b2ad2a9f3 P 7c7cf1bf258e42808c7808a7d9513838: Deleting consensus metadata
I20260812 17:51:45.296180 6393 catalog_manager.cc:4841] TS b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777): tablet 4ea821da2ae748859a01a17b2ad2a9f3 (table test-table [id=bf9b8eb3afb44524b2ed22557b954125]) successfully deleted
I20260812 17:51:45.296423 6393 catalog_manager.cc:4841] TS 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015): tablet 4ea821da2ae748859a01a17b2ad2a9f3 (table test-table [id=bf9b8eb3afb44524b2ed22557b954125]) successfully deleted
W20260812 17:51:45.303579 6389 connection.cc:535] client connection to 127.13.38.1:34899 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 17:51:45.303761 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 6707
W20260812 17:51:45.303812 6389 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 17:51:45.311007 7114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:51:45.310450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:34899 (local address 127.0.0.1:41692)
0812 17:51:45.310757 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 17:51:45.310871 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:34899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
W20260812 17:51:45.311301 6389 catalog_manager.cc:4584] TS f3118249b0f540efaefb0701970a08ff (127.13.38.1:34899): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ea821da2ae748859a01a17b2ad2a9f3: Network error: Client connection negotiation failed: client connection to 127.13.38.1:34899: connect: Connection refused (error 111)
I20260812 17:51:45.320329 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 6908
I20260812 17:51:45.337416 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 6366
I20260812 17:52:05.352370 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34877
--webserver_interface=127.13.38.62
--webserver_port=40675
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:34877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:05.485355 7171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:05.485721 7171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:05.485767 7171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:05.490288 7171 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 17:52:05.490381 7171 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 17:52:05.490397 7171 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 17:52:05.490419 7171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:05.490439 7171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:05.490459 7171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:05.490476 7171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:05.490558 7171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:05.497102 7171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:34877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:34877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=40675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:05.498467 7171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:05.499804 7171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:05.510887 7180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:05.510931 7182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:05.510874 7179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:05.510881 7171 server_base.cc:1034] running on GCE node
I20260812 17:52:05.512022 7171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:05.513468 7171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:05.514729 7171 hybrid_clock.cc:648] HybridClock initialized: now 1786557125514695 us; error 64 us; skew 500 ppm
Aug 12 17:52:05 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557125, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:05.524403 7171 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:52:05.525743 7171 webserver.cc:460] Webserver started at http://127.13.38.62:40675/ using document root <none> and password file <none>
I20260812 17:52:05.526448 7171 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:05.526562 7171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:05.531445 7171 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:52:05.534435 7192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:05.536127 7171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 17:52:05.536274 7171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "feab3fcd17b3406e96fdcc25b05be9cb"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "92061c8f9861fcea4dd4f24dc8f33b4d"
server_key_iv: "51094be76a4ccccdab21ad3241857029"
server_key_version: "encryptionkey@0"
I20260812 17:52:05.536911 7171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:05.562037 7171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:05.566432 7171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:05.566646 7171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:05.575024 7171 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:34877
I20260812 17:52:05.575383 7262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:34877 every 8 connection(s)
I20260812 17:52:05.576386 7171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:05.581441 7264 sys_catalog.cc:263] Verifying existing consensus state
I20260812 17:52:05.582937 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 7171
I20260812 17:52:05.583415 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:34899
--local_ip_for_outbound_sockets=127.13.38.1
--tserver_master_addrs=127.13.38.62:34877
--webserver_port=36377
--webserver_interface=127.13.38.1
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:52:05.583539 7264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Bootstrap starting.
I20260812 17:52:05.605093 7264 log.cc:824] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:05.617218 7264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 17:52:05.617676 7264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Bootstrap complete.
I20260812 17:52:05.622864 7264 raft_consensus.cc:348] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } }
I20260812 17:52:05.623413 7264 raft_consensus.cc:729] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: feab3fcd17b3406e96fdcc25b05be9cb, State: Initialized, Role: FOLLOWER
I20260812 17:52:05.624147 7264 consensus_queue.cc:260] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } }
I20260812 17:52:05.624370 7264 raft_consensus.cc:388] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:05.624480 7264 raft_consensus.cc:482] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:05.624619 7264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 2 FOLLOWER]: Advancing to term 3
I20260812 17:52:05.627045 7264 raft_consensus.cc:504] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } }
I20260812 17:52:05.627490 7264 leader_election.cc:302] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: feab3fcd17b3406e96fdcc25b05be9cb; no voters:
I20260812 17:52:05.628029 7264 leader_election.cc:288] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 17:52:05.628175 7274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 3 FOLLOWER]: Leader election won for term 3
I20260812 17:52:05.628460 7274 raft_consensus.cc:686] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [term 3 LEADER]: Becoming Leader. State: Replica: feab3fcd17b3406e96fdcc25b05be9cb, State: Running, Role: LEADER
I20260812 17:52:05.628842 7274 consensus_queue.cc:237] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } }
I20260812 17:52:05.629534 7264 sys_catalog.cc:564] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:05.630110 7274 sys_catalog.cc:455] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader feab3fcd17b3406e96fdcc25b05be9cb. Latest consensus state: current_term: 3 leader_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } } }
I20260812 17:52:05.630247 7274 sys_catalog.cc:458] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:05.630767 7279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:05.630733 7275 sys_catalog.cc:455] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feab3fcd17b3406e96fdcc25b05be9cb" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 34877 } } }
I20260812 17:52:05.631076 7275 sys_catalog.cc:458] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:05.637627 7279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:05.638655 7279 catalog_manager.cc:1259] Loaded cluster ID: c683fa90a27d4494ad348c14efcdb69a
I20260812 17:52:05.638784 7279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:05.641387 7279 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:05.642433 7279 catalog_manager.cc:5879] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Loaded TSK: 0
I20260812 17:52:05.642552 7279 catalog_manager.cc:5879] T 00000000000000000000000000000000 P feab3fcd17b3406e96fdcc25b05be9cb: Loaded TSK: 1
F20260812 17:52:05.643893 7279 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa2addcb9ad google::LogMessage::Fail() at ??:0
@ 0x7fa2addce88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa2addcb4de google::LogMessage::Flush() at ??:0
@ 0x7fa2addcca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa2b0a8f18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa2b0a87f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa2b87347bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa2b870e99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa2b870f178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fa2b8750df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fa2b876f82b std::function<>::operator()() at ??:0
@ 0x7fa2b870eb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fa2b870ff3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa2b870c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fa2b8750640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fa2b001d3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa2b001dc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fa2b001f3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fa2b000e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa2b16526db start_thread at ??:0
@ 0x7fa2ac8c961f clone at ??:0
W20260812 17:52:05.747905 7271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:05.748313 7271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:05.748389 7271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:05.755247 7271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:05.755339 7271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:05.755439 7271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:52:05.764935 7271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:34899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=36377
--tserver_master_addrs=127.13.38.62:34877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:05.766530 7271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:05.768011 7271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:05.781162 7365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:05.781196 7361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:05.781198 7362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:05.781765 7271 server_base.cc:1034] running on GCE node
I20260812 17:52:05.782356 7271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:05.783326 7271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:05.784564 7271 hybrid_clock.cc:648] HybridClock initialized: now 1786557125784524 us; error 57 us; skew 500 ppm
Aug 12 17:52:05 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557125, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:05.788770 7271 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:52:05.790184 7271 webserver.cc:460] Webserver started at http://127.13.38.1:36377/ using document root <none> and password file <none>
I20260812 17:52:05.791031 7271 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:05.791145 7271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:05.797159 7271 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:52:05.800666 7375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:05.802501 7271 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:52:05.802665 7271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "f3118249b0f540efaefb0701970a08ff"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "90ca39a0d1267a41f7f10991c64f1df6"
server_key_iv: "5032297ad3e1b410ca786ef7d32b50ac"
server_key_version: "encryptionkey@0"
I20260812 17:52:05.803556 7271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:05.834237 7271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:05.839071 7271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:05.839337 7271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:05.840271 7271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:05.841959 7385 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 17:52:05.846638 7271 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 17:52:05.846743 7271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260812 17:52:05.846823 7271 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 17:52:05.849426 7271 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 17:52:05.849519 7271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260812 17:52:05.849881 7385 tablet_bootstrap.cc:492] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff: Bootstrap starting.
I20260812 17:52:05.862933 7271 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:34899
I20260812 17:52:05.863283 7548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:34899 every 8 connection(s)
I20260812 17:52:05.864542 7271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:52:05.872341 7385 log.cc:824] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:05.874318 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 7271
I20260812 17:52:05.874866 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:39777
--local_ip_for_outbound_sockets=127.13.38.2
--tserver_master_addrs=127.13.38.62:34877
--webserver_port=35685
--webserver_interface=127.13.38.2
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:52:05.882489 7385 tablet_bootstrap.cc:492] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 17:52:05.883098 7385 tablet_bootstrap.cc:492] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff: Bootstrap complete.
I20260812 17:52:05.885106 7385 ts_tablet_manager.cc:1397] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff: Time spent bootstrapping tablet: real 0.036s user 0.026s sys 0.008s
I20260812 17:52:05.890271 7385 raft_consensus.cc:348] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:52:05.891122 7385 raft_consensus.cc:729] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3118249b0f540efaefb0701970a08ff, State: Initialized, Role: FOLLOWER
I20260812 17:52:05.891884 7385 consensus_queue.cc:260] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:52:05.893317 7385 ts_tablet_manager.cc:1428] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
W20260812 17:52:05.951690 7552 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:52:05.867252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:54651)
0812 17:52:05.867986 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:05.868055 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:05.869326 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:05.950624 (+ 81298us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":242,"threads_started":1}
W20260812 17:52:05.952356 7549 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:34877 (0 consecutive failures): Network error: Failed to ping master at 127.13.38.62:34877: Client connection negotiation failed: client connection to 127.13.38.62:34877: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 17:52:05.953315 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:05.952955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:45383)
0812 17:52:05.953149 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:05.953246 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260812 17:52:05.954154 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:05.953885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:48303)
0812 17:52:05.954026 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:05.954084 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 17:52:05.954423 7549 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 17:52:06.018716 7555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:06.019073 7555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:06.019124 7555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:06.023597 7555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:06.023685 7555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:06.023751 7555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:52:06.029712 7555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:39777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=35685
--tserver_master_addrs=127.13.38.62:34877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:06.031052 7555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:06.032611 7555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:06.044368 7571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:06.044395 7565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:06.044368 7566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:06.044783 7555 server_base.cc:1034] running on GCE node
I20260812 17:52:06.045544 7555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:06.046610 7555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:06.047812 7555 hybrid_clock.cc:648] HybridClock initialized: now 1786557126047763 us; error 67 us; skew 500 ppm
Aug 12 17:52:06 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557126, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:06.051086 7555 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:52:06.052285 7555 webserver.cc:460] Webserver started at http://127.13.38.2:35685/ using document root <none> and password file <none>
I20260812 17:52:06.052992 7555 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:06.053066 7555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:06.058143 7555 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:52:06.062156 7580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:06.063836 7555 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 17:52:06.063973 7555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "b49e090a6a244faab880eeb69dd73de6"
format_stamp: "Formatted at 2026-08-12 17:51:22 on dist-test-slave-sb2z"
server_key: "797ad95b938e56c8c68be10b1d34f7a1"
server_key_iv: "b00968eccbba05543092b94317e62aeb"
server_key_version: "encryptionkey@0"
I20260812 17:52:06.064611 7555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:06.084014 7555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:06.088152 7555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:06.088339 7555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:06.088956 7555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:06.089843 7555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:06.089897 7555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:06.089942 7555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:06.089958 7555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:06.100430 7555 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:39777
I20260812 17:52:06.100458 7759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:39777 every 8 connection(s)
I20260812 17:52:06.101670 7555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:52:06.102734 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 7555
I20260812 17:52:06.103178 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:34015
--local_ip_for_outbound_sockets=127.13.38.3
--tserver_master_addrs=127.13.38.62:34877
--webserver_port=38647
--webserver_interface=127.13.38.3
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:52:06.107823 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:06.105668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:44251)
0812 17:52:06.106231 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:06.106379 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":156,"threads_started":1}
W20260812 17:52:06.108495 7760 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:34877 (0 consecutive failures): Network error: Failed to ping master at 127.13.38.62:34877: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
I20260812 17:52:06.109458 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:06.109048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:48943)
0812 17:52:06.109321 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:06.109382 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260812 17:52:06.110406 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:06.110138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:53463)
0812 17:52:06.110287 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:06.110342 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 17:52:06.110669 7760 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 17:52:06.230327 7763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:06.230633 7763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:06.230679 7763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:06.235081 7763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:06.235183 7763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:06.235249 7763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:52:06.241948 7763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:34015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=38647
--tserver_master_addrs=127.13.38.62:34877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:06.243238 7763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:06.244445 7763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:06.255194 7770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:06.255210 7771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:06.255205 7774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:06.255399 7763 server_base.cc:1034] running on GCE node
I20260812 17:52:06.255966 7763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:06.256613 7763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:06.257822 7763 hybrid_clock.cc:648] HybridClock initialized: now 1786557126257780 us; error 52 us; skew 500 ppm
Aug 12 17:52:06 dist-test-slave-sb2z krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557126, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:06.268915 7763 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:52:06.270233 7763 webserver.cc:460] Webserver started at http://127.13.38.3:38647/ using document root <none> and password file <none>
I20260812 17:52:06.270917 7763 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:06.271026 7763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:06.275584 7763 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:06.278296 7782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:06.279910 7763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:06.280136 7763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "7c7cf1bf258e42808c7808a7d9513838"
format_stamp: "Formatted at 2026-08-12 17:51:23 on dist-test-slave-sb2z"
server_key: "00996d975a4d29b0a41a55aa31af4edb"
server_key_iv: "7e3c43c7f2020f223c2595b650e105c2"
server_key_version: "encryptionkey@0"
I20260812 17:52:06.280807 7763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:06.303850 7763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:06.308463 7763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:06.308761 7763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:06.309777 7763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:06.312888 7763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:06.312985 7763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:06.313058 7763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:06.313088 7763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:06.330909 7763 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:34015
I20260812 17:52:06.330950 7958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:34015 every 8 connection(s)
I20260812 17:52:06.332279 7763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:52:06.335763 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 7763
I20260812 17:52:06.341068 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:06.338221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:59619)
0812 17:52:06.338859 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:06.339047 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":124,"threads_started":1}
W20260812 17:52:06.341732 7959 heartbeater.cc:644] Failed to heartbeat to 127.13.38.62:34877 (0 consecutive failures): Network error: Failed to ping master at 127.13.38.62:34877: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
I20260812 17:52:06.342682 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:06.342302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:48701)
0812 17:52:06.342467 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:06.342534 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 17:52:06.343653 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:06.343241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:39969)
0812 17:52:06.343427 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:06.343492 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260812 17:52:06.346053 7959 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 17:52:06.955935 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:06.955282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:60811)
0812 17:52:06.955733 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:06.955812 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":89,"threads_started":1}
I20260812 17:52:07.112033 7964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:07.111369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:46677)
0812 17:52:07.111828 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:07.111900 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":85,"threads_started":1}
I20260812 17:52:07.154047 7968 raft_consensus.cc:482] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:52:07.154320 7968 raft_consensus.cc:504] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:52:07.155884 7968 leader_election.cc:288] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777), 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015)
Aug 12 17:52:07 dist-test-slave-sb2z krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557125, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.3@KRBTEST.COM
Aug 12 17:52:07 dist-test-slave-sb2z krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557125, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.2@KRBTEST.COM
I20260812 17:52:07.172541 7974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:07.156926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:47615 (local address 127.13.38.3:34015)
0812 17:52:07.157389 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:07.157401 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:07.157436 (+ 35us) server_negotiation.cc:407] Connection header received
0812 17:52:07.158108 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:07.158114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:07.158742 (+ 628us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:07.159163 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:07.160564 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:07.161651 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:07.163184 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:07.163786 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:07.163967 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:07.167181 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:07.167188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:07.167204 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:07.167241 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:07.169385 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:07.170066 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:07.170074 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:07.170081 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:07.170244 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:07.170653 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:07.170656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:07.170658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:07.171168 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:07.171330 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:07.172027 (+ 697us) server_negotiation.cc:299] Negotiation successful
0812 17:52:07.172316 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":100,"threads_started":1}
I20260812 17:52:07.172524 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:07.156691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:34015 (local address 127.13.38.1:47615)
0812 17:52:07.157057 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:07.157076 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:07.157205 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:07.159101 (+ 1896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:07.159105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:07.159727 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:07.160268 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:07.160285 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:07.161866 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:07.161871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:07.162985 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:07.162997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:07.163943 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:07.163948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:07.164230 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:07.165111 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:52:07.165130 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:07.166853 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:07.169569 (+ 2716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:07.169576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:07.169594 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:07.169895 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:07.170360 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:07.170364 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:07.170366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:07.170536 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:07.171303 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:07.171310 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:07.171674 (+ 364us) client_negotiation.cc:769] Sending connection context
0812 17:52:07.171892 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 17:52:07.172315 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":268,"spinlock_wait_cycles":32512,"thread_start_us":94,"threads_started":1}
I20260812 17:52:07.172541 7971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:07.156524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:58197 (local address 127.13.38.2:39777)
0812 17:52:07.157023 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:07.157051 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:07.157106 (+ 55us) server_negotiation.cc:407] Connection header received
0812 17:52:07.157951 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:07.157960 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:07.158386 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:07.158711 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:07.160506 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:07.161655 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:07.163117 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:07.163801 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:07.164003 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:07.167520 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:07.167569 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:07.167589 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:07.167636 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:07.169735 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:07.170286 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:07.170291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:07.170295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:07.170420 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:07.170742 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:07.170746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:07.170747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:07.171166 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:07.171326 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:07.172019 (+ 693us) server_negotiation.cc:299] Negotiation successful
0812 17:52:07.172329 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":134,"threads_started":1}
I20260812 17:52:07.172570 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:07.156335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:39777 (local address 127.13.38.1:58197)
0812 17:52:07.156516 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:07.156534 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:07.156669 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:07.158970 (+ 2301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:07.158979 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:07.159727 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:07.160267 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:07.160284 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:07.161849 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:07.161852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:07.162938 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:07.162950 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:07.163962 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:07.163966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:07.164450 (+ 484us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:07.165342 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:52:07.165355 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:07.167236 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:07.169910 (+ 2674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:07.169915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:07.169917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:07.170137 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:07.170536 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:07.170539 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:07.170541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:07.170639 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:07.171301 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:07.171308 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:07.171671 (+ 363us) client_negotiation.cc:769] Sending connection context
0812 17:52:07.171892 (+ 221us) client_negotiation.cc:240] Negotiation successful
0812 17:52:07.172412 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":682,"spinlock_wait_cycles":32640}
W20260812 17:52:07.176189 7381 leader_election.cc:341] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777): Not found: Tablet not found: 4ea821da2ae748859a01a17b2ad2a9f3
W20260812 17:52:07.176535 7381 leader_election.cc:341] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015): Not found: Tablet not found: 4ea821da2ae748859a01a17b2ad2a9f3
I20260812 17:52:07.176623 7381 leader_election.cc:302] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3118249b0f540efaefb0701970a08ff; no voters: 7c7cf1bf258e42808c7808a7d9513838, b49e090a6a244faab880eeb69dd73de6
I20260812 17:52:07.176963 7968 raft_consensus.cc:2726] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 17:52:07.347342 7975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:07.346816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:34025)
0812 17:52:07.347149 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:07.347207 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":106,"threads_started":1}
I20260812 17:52:07.957643 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:07.957131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:60107)
0812 17:52:07.957457 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:07.957528 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":122,"threads_started":1}
I20260812 17:52:08.113657 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:08.113136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:41835)
0812 17:52:08.113479 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:08.113548 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":86,"threads_started":1}
I20260812 17:52:08.348929 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:08.348424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:37947)
0812 17:52:08.348737 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:08.348804 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":90,"threads_started":1}
I20260812 17:52:08.959143 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:08.958592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:37591)
0812 17:52:08.958927 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:08.959033 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":98,"threads_started":1}
I20260812 17:52:09.083477 7980 raft_consensus.cc:482] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:52:09.083719 7980 raft_consensus.cc:504] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:52:09.084231 7980 leader_election.cc:288] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777), 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015)
W20260812 17:52:09.085326 7381 leader_election.cc:341] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015): Not found: Tablet not found: 4ea821da2ae748859a01a17b2ad2a9f3
W20260812 17:52:09.085481 7381 leader_election.cc:341] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777): Not found: Tablet not found: 4ea821da2ae748859a01a17b2ad2a9f3
I20260812 17:52:09.085530 7381 leader_election.cc:302] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3118249b0f540efaefb0701970a08ff; no voters: 7c7cf1bf258e42808c7808a7d9513838, b49e090a6a244faab880eeb69dd73de6
I20260812 17:52:09.085686 7980 raft_consensus.cc:2726] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 17:52:09.115094 7983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:09.114610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:57585)
0812 17:52:09.114886 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:09.114944 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":81,"threads_started":1}
I20260812 17:52:09.350559 7984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:09.349978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:58949)
0812 17:52:09.350366 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:09.350444 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":94,"threads_started":1}
I20260812 17:52:09.960870 7985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:09.960358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:40281)
0812 17:52:09.960692 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:09.960759 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":118,"threads_started":1}
I20260812 17:52:10.116812 7986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:10.116268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:37761)
0812 17:52:10.116623 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:10.116700 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":91,"threads_started":1}
I20260812 17:52:10.352355 7987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:10.351808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:41747)
0812 17:52:10.352175 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:10.352244 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":104,"threads_started":1}
I20260812 17:52:10.962935 7988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:10.962194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:51151)
0812 17:52:10.962569 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:10.962814 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":93,"threads_started":1}
I20260812 17:52:11.118561 7989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:11.117977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:41017)
0812 17:52:11.118378 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:11.118447 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":148,"threads_started":1}
I20260812 17:52:11.354017 7990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:11.353436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:34975)
0812 17:52:11.353824 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:11.353897 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":101,"threads_started":1}
I20260812 17:52:11.897781 7992 raft_consensus.cc:482] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:52:11.897925 7992 raft_consensus.cc:504] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:52:11.898357 7992 leader_election.cc:288] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777), 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015)
W20260812 17:52:11.899307 7381 leader_election.cc:341] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777): Not found: Tablet not found: 4ea821da2ae748859a01a17b2ad2a9f3
W20260812 17:52:11.899449 7381 leader_election.cc:341] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015): Not found: Tablet not found: 4ea821da2ae748859a01a17b2ad2a9f3
I20260812 17:52:11.899489 7381 leader_election.cc:302] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3118249b0f540efaefb0701970a08ff; no voters: 7c7cf1bf258e42808c7808a7d9513838, b49e090a6a244faab880eeb69dd73de6
I20260812 17:52:11.899720 7992 raft_consensus.cc:2726] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 17:52:11.964527 7993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:11.963976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:52235)
0812 17:52:11.964337 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:11.964410 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":93,"threads_started":1}
I20260812 17:52:12.120146 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:12.119659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:35949)
0812 17:52:12.119961 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:12.120035 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":82,"threads_started":1}
I20260812 17:52:12.355626 7996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:12.355026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:54321)
0812 17:52:12.355357 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:12.355427 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":83,"threads_started":1}
I20260812 17:52:12.966030 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:12.965488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:40609)
0812 17:52:12.965796 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:12.965862 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":93,"threads_started":1}
I20260812 17:52:13.121852 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:13.121306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:39507)
0812 17:52:13.121693 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:13.121752 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":103,"threads_started":1}
I20260812 17:52:13.357352 8001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:13.356759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:50165)
0812 17:52:13.357169 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:13.357246 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":105,"threads_started":1}
I20260812 17:52:13.967757 8002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:13.967178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:42287)
0812 17:52:13.967568 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:13.967645 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":149,"threads_started":1}
I20260812 17:52:14.123360 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:14.122846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:38479)
0812 17:52:14.123187 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:14.123256 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":109,"threads_started":1}
I20260812 17:52:14.359071 8006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:14.358483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:35777)
0812 17:52:14.358865 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:14.358953 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":115,"threads_started":1}
I20260812 17:52:14.969412 8007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:14.968781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:55353)
0812 17:52:14.969236 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:14.969305 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":121,"threads_started":1}
I20260812 17:52:15.125109 8008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:15.124510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:33505)
0812 17:52:15.124877 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:15.124955 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":93,"threads_started":1}
I20260812 17:52:15.360834 8010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:15.360318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:52987)
0812 17:52:15.360643 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:15.360717 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":92,"threads_started":1}
I20260812 17:52:15.971064 8012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:15.970577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:48601)
0812 17:52:15.970897 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:15.970961 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":88,"threads_started":1}
I20260812 17:52:16.109205 8013 raft_consensus.cc:482] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:52:16.109364 8013 raft_consensus.cc:504] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:52:16.109918 8013 leader_election.cc:288] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777), 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015)
W20260812 17:52:16.110948 7381 leader_election.cc:341] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777): Not found: Tablet not found: 4ea821da2ae748859a01a17b2ad2a9f3
W20260812 17:52:16.111173 7381 leader_election.cc:341] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015): Not found: Tablet not found: 4ea821da2ae748859a01a17b2ad2a9f3
I20260812 17:52:16.111238 7381 leader_election.cc:302] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3118249b0f540efaefb0701970a08ff; no voters: 7c7cf1bf258e42808c7808a7d9513838, b49e090a6a244faab880eeb69dd73de6
I20260812 17:52:16.111434 8013 raft_consensus.cc:2726] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 17:52:16.126540 8014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:16.126063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:46387)
0812 17:52:16.126364 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:16.126423 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":108,"threads_started":1}
I20260812 17:52:16.362363 8015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:16.361903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:44721)
0812 17:52:16.362195 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:16.362258 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":100,"threads_started":1}
I20260812 17:52:16.972621 8018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:16.972095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:44587)
0812 17:52:16.972447 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:16.972514 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":101,"threads_started":1}
I20260812 17:52:17.128142 8019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:17.127602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:48133)
0812 17:52:17.127938 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:17.128016 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":102,"threads_started":1}
I20260812 17:52:17.364125 8022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:17.363419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:38313)
0812 17:52:17.363885 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:17.364012 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":254,"threads_started":1}
I20260812 17:52:17.974232 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:17.973709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:55537)
0812 17:52:17.974044 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:17.974123 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":92,"threads_started":1}
I20260812 17:52:18.129794 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:18.129232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:36409)
0812 17:52:18.129606 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:18.129681 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":114,"threads_started":1}
I20260812 17:52:18.365921 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:18.365356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:34459)
0812 17:52:18.365730 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:18.365801 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":146,"threads_started":1}
I20260812 17:52:18.975903 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:18.975227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:45719)
0812 17:52:18.975609 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:18.975792 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":95,"threads_started":1}
I20260812 17:52:19.032353 8033 raft_consensus.cc:482] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:52:19.032558 8033 raft_consensus.cc:504] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49e090a6a244faab880eeb69dd73de6" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 39777 } } peers { permanent_uuid: "7c7cf1bf258e42808c7808a7d9513838" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 34015 } } peers { permanent_uuid: "f3118249b0f540efaefb0701970a08ff" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 34899 } }
I20260812 17:52:19.032989 8033 leader_election.cc:288] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777), 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015)
W20260812 17:52:19.033938 7381 leader_election.cc:341] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b49e090a6a244faab880eeb69dd73de6 (127.13.38.2:39777): Not found: Tablet not found: 4ea821da2ae748859a01a17b2ad2a9f3
W20260812 17:52:19.034195 7381 leader_election.cc:341] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c7cf1bf258e42808c7808a7d9513838 (127.13.38.3:34015): Not found: Tablet not found: 4ea821da2ae748859a01a17b2ad2a9f3
I20260812 17:52:19.034257 7381 leader_election.cc:302] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3118249b0f540efaefb0701970a08ff; no voters: 7c7cf1bf258e42808c7808a7d9513838, b49e090a6a244faab880eeb69dd73de6
I20260812 17:52:19.034494 8033 raft_consensus.cc:2726] T 4ea821da2ae748859a01a17b2ad2a9f3 P f3118249b0f540efaefb0701970a08ff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 17:52:19.131386 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:19.130883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:48361)
0812 17:52:19.131196 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:19.131269 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":97,"threads_started":1}
I20260812 17:52:19.367408 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:19.366899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:39519)
0812 17:52:19.367230 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:19.367299 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":101,"threads_started":1}
I20260812 17:52:19.977542 8040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:19.977002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:51579)
0812 17:52:19.977355 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:19.977433 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":116,"threads_started":1}
I20260812 17:52:20.132975 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:20.132428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:43497)
0812 17:52:20.132784 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:20.132854 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":124,"threads_started":1}
I20260812 17:52:20.368880 8044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:20.368379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:48505)
0812 17:52:20.368701 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:20.368770 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":93,"threads_started":1}
I20260812 17:52:20.979432 8046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:20.978901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.1:42345)
0812 17:52:20.979248 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:20.979306 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":158,"threads_started":1}
I20260812 17:52:21.134655 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:21.134151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.2:42983)
0812 17:52:21.134477 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:21.134548 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":85,"threads_started":1}
I20260812 17:52:21.338227 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 7271
I20260812 17:52:21.360025 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 7555
I20260812 17:52:21.370544 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:21.370086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:34877 (local address 127.13.38.3:55125)
0812 17:52:21.370364 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:21.370434 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.62:34877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":100,"threads_started":1}
I20260812 17:52:21.375156 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 7763
2026-08-12T17:52:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60560 ms)
[----------] 30 tests from SecurityITest (260934 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:21 dist-test-slave-sb2z krb5kdc[8053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:21 dist-test-slave-sb2z krb5kdc[8053](info): set up 2 sockets
Aug 12 17:52:21 dist-test-slave-sb2z krb5kdc[8053](info): commencing operation
krb5kdc: starting...
W20260812 17:52:22.723830 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:22 dist-test-slave-sb2z krb5kdc[8053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:52:22.880537 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:40115
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:39781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:40115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:23.006861 8070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:23.007256 8070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:23.007351 8070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:23.011562 8070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:23.011680 8070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:23.011718 8070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:23.011765 8070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:23.011795 8070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:23.018339 8070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:40115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:40115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:23.019876 8070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:23.021231 8070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:23.031303 8082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:23.031334 8085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:23.031338 8081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:23.031775 8070 server_base.cc:1034] running on GCE node
I20260812 17:52:23.032603 8070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:23.033950 8070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:23.035184 8070 hybrid_clock.cc:648] HybridClock initialized: now 1786557143035141 us; error 45 us; skew 500 ppm
Aug 12 17:52:23 dist-test-slave-sb2z krb5kdc[8053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557143, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:23.039577 8070 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:52:23.040746 8070 webserver.cc:460] Webserver started at http://127.13.38.62:35895/ using document root <none> and password file <none>
I20260812 17:52:23.041464 8070 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:23.041571 8070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:23.041819 8070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:23.044291 8070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "d78d9b5b228e4bbea8d21ce80561d2a5"
format_stamp: "Formatted at 2026-08-12 17:52:23 on dist-test-slave-sb2z"
server_key: "11882ea92d680abf8a173524312b2930"
server_key_iv: "f287eb301cf7739c25722385bc53678e"
server_key_version: "encryptionkey@0"
I20260812 17:52:23.044957 8070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "d78d9b5b228e4bbea8d21ce80561d2a5"
format_stamp: "Formatted at 2026-08-12 17:52:23 on dist-test-slave-sb2z"
server_key: "11882ea92d680abf8a173524312b2930"
server_key_iv: "f287eb301cf7739c25722385bc53678e"
server_key_version: "encryptionkey@0"
I20260812 17:52:23.049316 8070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:52:23.052493 8096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:23.054486 8070 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:52:23.054720 8070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "d78d9b5b228e4bbea8d21ce80561d2a5"
format_stamp: "Formatted at 2026-08-12 17:52:23 on dist-test-slave-sb2z"
server_key: "11882ea92d680abf8a173524312b2930"
server_key_iv: "f287eb301cf7739c25722385bc53678e"
server_key_version: "encryptionkey@0"
I20260812 17:52:23.054891 8070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:23.076057 8070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:23.079679 8070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:23.079942 8070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:23.088580 8070 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:40115
I20260812 17:52:23.088600 8180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:40115 every 8 connection(s)
I20260812 17:52:23.089692 8070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:23.091879 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 8070
I20260812 17:52:23.092064 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:23.092391 13464 external_mini_cluster.cc:1428] Setting key 3ba2048307422095a03d1f0e1b01031a
I20260812 17:52:23.093405 8181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:23.100924 8181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5: Bootstrap starting.
Aug 12 17:52:23 dist-test-slave-sb2z krb5kdc[8053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:52:23.103493 8181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:23.104578 8181 log.cc:824] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:23.107221 8181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5: No bootstrap required, opened a new log
I20260812 17:52:23.109982 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:23.093944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47764 (local address 127.13.38.62:40115)
0812 17:52:23.094429 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:23.094450 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:23.094506 (+ 56us) server_negotiation.cc:407] Connection header received
0812 17:52:23.095428 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:23.095454 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:23.095950 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:23.096351 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:23.097261 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:23.098793 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:23.099875 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:23.100580 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:23.100745 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:23.103408 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:23.103419 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:23.103439 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:23.103489 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:23.105460 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:23.105970 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:23.105977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:23.105984 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:23.106128 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:23.106403 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:23.106407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:23.106409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:23.106951 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:23.107151 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:23.107780 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 17:52:23.108605 (+ 825us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":138,"threads_started":1}
I20260812 17:52:23.112097 8181 raft_consensus.cc:348] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78d9b5b228e4bbea8d21ce80561d2a5" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40115 } }
I20260812 17:52:23.112361 8181 raft_consensus.cc:374] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:23.112399 8181 raft_consensus.cc:729] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78d9b5b228e4bbea8d21ce80561d2a5, State: Initialized, Role: FOLLOWER
I20260812 17:52:23.112964 8181 consensus_queue.cc:260] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78d9b5b228e4bbea8d21ce80561d2a5" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40115 } }
I20260812 17:52:23.113147 8181 raft_consensus.cc:388] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:23.113240 8181 raft_consensus.cc:482] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:23.113346 8181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:23.114507 8181 raft_consensus.cc:504] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78d9b5b228e4bbea8d21ce80561d2a5" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40115 } }
I20260812 17:52:23.114904 8181 leader_election.cc:302] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d78d9b5b228e4bbea8d21ce80561d2a5; no voters:
I20260812 17:52:23.115345 8181 leader_election.cc:288] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:23.115410 8190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:23.115684 8190 raft_consensus.cc:686] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [term 1 LEADER]: Becoming Leader. State: Replica: d78d9b5b228e4bbea8d21ce80561d2a5, State: Running, Role: LEADER
I20260812 17:52:23.116068 8190 consensus_queue.cc:237] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78d9b5b228e4bbea8d21ce80561d2a5" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40115 } }
I20260812 17:52:23.116847 8181 sys_catalog.cc:564] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:23.118117 8192 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d78d9b5b228e4bbea8d21ce80561d2a5. Latest consensus state: current_term: 1 leader_uuid: "d78d9b5b228e4bbea8d21ce80561d2a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78d9b5b228e4bbea8d21ce80561d2a5" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40115 } } }
I20260812 17:52:23.118129 8191 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d78d9b5b228e4bbea8d21ce80561d2a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78d9b5b228e4bbea8d21ce80561d2a5" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40115 } } }
I20260812 17:52:23.118389 8192 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:23.118398 8191 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:23.118940 8203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:23.122514 8203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:23.129940 8203 catalog_manager.cc:1347] Generated new cluster ID: 70ab6a3a81774da1a2c5c9dbb925ccc4
I20260812 17:52:23.130056 8203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:23.151011 8203 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:23.152287 8203 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:23.160498 8203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5: Generated new TSK 0
I20260812 17:52:23.161419 8203 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:52:23.212481 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:40115
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:39781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:23.348483 8220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:23.348809 8220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:23.348857 8220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:23.353305 8220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:23.353401 8220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:23.353473 8220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:52:23.359697 8220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:40115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:23.361040 8220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:23.362275 8220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:23.373086 8232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:23.373064 8234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:23.373162 8231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:23.373256 8220 server_base.cc:1034] running on GCE node
I20260812 17:52:23.374060 8220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:23.374795 8220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:23.375993 8220 hybrid_clock.cc:648] HybridClock initialized: now 1786557143375973 us; error 32 us; skew 500 ppm
Aug 12 17:52:23 dist-test-slave-sb2z krb5kdc[8053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557143, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:23.379261 8220 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:52:23.380657 8220 webserver.cc:460] Webserver started at http://127.13.38.1:38603/ using document root <none> and password file <none>
I20260812 17:52:23.381364 8220 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:23.381441 8220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:23.381695 8220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:23.384090 8220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "b2f5acb303ba41e5b0842e5a77bc2425"
format_stamp: "Formatted at 2026-08-12 17:52:23 on dist-test-slave-sb2z"
server_key: "21226667b11d1ed2836ce631b2ab1229"
server_key_iv: "ee5f098f16c31fa230a2530d4d6673bd"
server_key_version: "encryptionkey@0"
I20260812 17:52:23.384738 8220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "b2f5acb303ba41e5b0842e5a77bc2425"
format_stamp: "Formatted at 2026-08-12 17:52:23 on dist-test-slave-sb2z"
server_key: "21226667b11d1ed2836ce631b2ab1229"
server_key_iv: "ee5f098f16c31fa230a2530d4d6673bd"
server_key_version: "encryptionkey@0"
I20260812 17:52:23.389348 8220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:52:23.392563 8248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:23.394125 8220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:52:23.394297 8220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "b2f5acb303ba41e5b0842e5a77bc2425"
format_stamp: "Formatted at 2026-08-12 17:52:23 on dist-test-slave-sb2z"
server_key: "21226667b11d1ed2836ce631b2ab1229"
server_key_iv: "ee5f098f16c31fa230a2530d4d6673bd"
server_key_version: "encryptionkey@0"
I20260812 17:52:23.394438 8220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:23.432952 8220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:23.437089 8220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:23.437337 8220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:23.438031 8220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:23.438985 8220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:23.439059 8220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:23.439136 8220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:23.439183 8220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:23.450408 8220 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:40893
I20260812 17:52:23.450430 8425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:40893 every 8 connection(s)
I20260812 17:52:23.451498 8220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:52:23.452495 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 8220
I20260812 17:52:23.452656 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:52:23.453051 13464 external_mini_cluster.cc:1428] Setting key 0b084c4d9b3734f8a946cc1b98813803
Aug 12 17:52:23 dist-test-slave-sb2z krb5kdc[8053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557143, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
I20260812 17:52:23.470723 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:23.453809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:55725 (local address 127.13.38.62:40115)
0812 17:52:23.453987 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:23.453994 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:23.455516 (+ 1522us) server_negotiation.cc:407] Connection header received
0812 17:52:23.457968 (+ 2452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:23.457972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:23.458061 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:23.458176 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:23.459858 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:23.460506 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:23.461970 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:23.462237 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:23.462363 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:23.465770 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:23.465774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:23.465776 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:23.465807 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:23.467645 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:23.468404 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:23.468408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:23.468410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:23.468506 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:23.469013 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:23.469022 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:23.469023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:23.469253 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:23.469377 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:23.470279 (+ 902us) server_negotiation.cc:299] Negotiation successful
0812 17:52:23.470457 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 17:52:23.471690 8428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:23.454418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:40115 (local address 127.13.38.1:55725)
0812 17:52:23.455258 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:23.455332 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:23.457630 (+ 2298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:23.458410 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:23.458419 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:23.458934 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:23.459607 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:23.459625 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:23.460666 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:23.460670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:23.461812 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:23.461824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:23.462431 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:23.462436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:23.462637 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:23.463470 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:52:23.463489 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:23.465547 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:23.467799 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:23.467806 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:23.467833 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:23.468279 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:23.468632 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:23.468636 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:23.468638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:23.468893 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:23.469377 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:23.469387 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:23.469844 (+ 457us) client_negotiation.cc:769] Sending connection context
0812 17:52:23.470102 (+ 258us) client_negotiation.cc:240] Negotiation successful
0812 17:52:23.470465 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":184,"threads_started":1}
I20260812 17:52:23.473433 8426 heartbeater.cc:347] Connected to a master server at 127.13.38.62:40115
I20260812 17:52:23.473747 8426 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:23.474412 8426 heartbeater.cc:511] Master 127.13.38.62:40115 requested a full tablet report, sending...
I20260812 17:52:23.476336 8129 ts_manager.cc:194] Registered new tserver with Master: b2f5acb303ba41e5b0842e5a77bc2425 (127.13.38.1:40893)
I20260812 17:52:23.477744 8129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:55725
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:52:23.492956 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:40115
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:39781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:23.627956 8431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:23.628288 8431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:23.628341 8431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:23.633293 8431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:23.633387 8431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:23.633457 8431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:52:23.640478 8431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:40115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:23.641857 8431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:23.643100 8431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:23.654939 8440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:23.654982 8431 server_base.cc:1034] running on GCE node
W20260812 17:52:23.654863 8439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:23.655187 8442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:23.655774 8431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:23.656512 8431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:23.657706 8431 hybrid_clock.cc:648] HybridClock initialized: now 1786557143657687 us; error 36 us; skew 500 ppm
Aug 12 17:52:23 dist-test-slave-sb2z krb5kdc[8053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557143, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:23.661800 8431 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:52:23.662864 8431 webserver.cc:460] Webserver started at http://127.13.38.2:45661/ using document root <none> and password file <none>
I20260812 17:52:23.663470 8431 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:23.663600 8431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:23.663818 8431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:23.666021 8431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "083c8ca907e54287a386fc144e800ec1"
format_stamp: "Formatted at 2026-08-12 17:52:23 on dist-test-slave-sb2z"
server_key: "6eeb9274fa9a500465e64ffdf39a3658"
server_key_iv: "3841a46e58765e09c6eeca4cb9543313"
server_key_version: "encryptionkey@0"
I20260812 17:52:23.666596 8431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "083c8ca907e54287a386fc144e800ec1"
format_stamp: "Formatted at 2026-08-12 17:52:23 on dist-test-slave-sb2z"
server_key: "6eeb9274fa9a500465e64ffdf39a3658"
server_key_iv: "3841a46e58765e09c6eeca4cb9543313"
server_key_version: "encryptionkey@0"
I20260812 17:52:23.670958 8431 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 17:52:23.674233 8453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:23.675928 8431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 17:52:23.676070 8431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "083c8ca907e54287a386fc144e800ec1"
format_stamp: "Formatted at 2026-08-12 17:52:23 on dist-test-slave-sb2z"
server_key: "6eeb9274fa9a500465e64ffdf39a3658"
server_key_iv: "3841a46e58765e09c6eeca4cb9543313"
server_key_version: "encryptionkey@0"
I20260812 17:52:23.676189 8431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:23.700901 8431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:23.704685 8431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:23.704867 8431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:23.705482 8431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:23.706435 8431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:23.706514 8431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:23.706584 8431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:23.706629 8431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:23.717752 8431 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:42177
I20260812 17:52:23.717823 8615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:42177 every 8 connection(s)
I20260812 17:52:23.718964 8431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:52:23.722601 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 8431
I20260812 17:52:23.722785 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:52:23.723186 13464 external_mini_cluster.cc:1428] Setting key 44c1b85ed0b07a2e4fcc65d7d9b01c72
Aug 12 17:52:23 dist-test-slave-sb2z krb5kdc[8053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557143, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:52:23.737413 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:23.721426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:33573 (local address 127.13.38.62:40115)
0812 17:52:23.721580 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:23.721585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:23.722816 (+ 1231us) server_negotiation.cc:407] Connection header received
0812 17:52:23.724972 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:23.724976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:23.725033 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:23.725123 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:23.726648 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:23.727312 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:23.728957 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:23.729365 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:23.729523 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:23.732569 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:23.732577 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:23.732583 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:23.732625 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:23.734629 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:23.735320 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:23.735327 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:23.735330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:23.735448 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:23.736067 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:23.736076 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:23.736078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:23.736288 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:23.736388 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:23.737143 (+ 755us) server_negotiation.cc:299] Negotiation successful
0812 17:52:23.737279 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 17:52:23.738490 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:23.721862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:40115 (local address 127.13.38.2:33573)
0812 17:52:23.722596 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:23.722673 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:23.724682 (+ 2009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:23.725356 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:23.725366 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:23.725850 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:23.726454 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:23.726473 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:23.727500 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:23.727504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:23.728773 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:23.728789 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:23.729615 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:23.729619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:23.729797 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:23.730533 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:52:23.730552 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:23.732324 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:23.734812 (+ 2488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:23.734820 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:23.734837 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:23.735162 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:23.735638 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:23.735641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:23.735643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:23.735924 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:23.736430 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:23.736437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:23.736757 (+ 320us) client_negotiation.cc:769] Sending connection context
0812 17:52:23.736968 (+ 211us) client_negotiation.cc:240] Negotiation successful
0812 17:52:23.737406 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":150,"threads_started":1}
I20260812 17:52:23.740137 8616 heartbeater.cc:347] Connected to a master server at 127.13.38.62:40115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
I20260812 17:52:23.740458 8616 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:23.741076 8616 heartbeater.cc:511] Master 127.13.38.62:40115 requested a full tablet report, sending...
I20260812 17:52:23.742520 8129 ts_manager.cc:194] Registered new tserver with Master: 083c8ca907e54287a386fc144e800ec1 (127.13.38.2:42177)
I20260812 17:52:23.743248 8129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:33573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:52:23.762883 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:40115
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:39781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:23.901888 8622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:23.902204 8622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:23.902251 8622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:23.906651 8622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:23.906746 8622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:23.906812 8622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:52:23.912703 8622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:40115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:23.913986 8622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:23.915148 8622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:23.926851 8628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:23.926853 8634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:23.926858 8629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:23.927194 8622 server_base.cc:1034] running on GCE node
I20260812 17:52:23.927806 8622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:23.928560 8622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:23.929777 8622 hybrid_clock.cc:648] HybridClock initialized: now 1786557143929732 us; error 58 us; skew 500 ppm
Aug 12 17:52:23 dist-test-slave-sb2z krb5kdc[8053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557143, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:23.933256 8622 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:52:23.934525 8622 webserver.cc:460] Webserver started at http://127.13.38.3:37585/ using document root <none> and password file <none>
I20260812 17:52:23.935279 8622 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:23.935386 8622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:23.935698 8622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:23.938022 8622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "d38a9d35e943444a94a31ed66a837efb"
format_stamp: "Formatted at 2026-08-12 17:52:23 on dist-test-slave-sb2z"
server_key: "d7934c5cc7005f759f9b3938ebcb241f"
server_key_iv: "41a74c875b91074d4316de114ee11915"
server_key_version: "encryptionkey@0"
I20260812 17:52:23.938688 8622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "d38a9d35e943444a94a31ed66a837efb"
format_stamp: "Formatted at 2026-08-12 17:52:23 on dist-test-slave-sb2z"
server_key: "d7934c5cc7005f759f9b3938ebcb241f"
server_key_iv: "41a74c875b91074d4316de114ee11915"
server_key_version: "encryptionkey@0"
I20260812 17:52:23.943331 8622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 17:52:23.946329 8642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:23.948182 8622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:23.948350 8622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "d38a9d35e943444a94a31ed66a837efb"
format_stamp: "Formatted at 2026-08-12 17:52:23 on dist-test-slave-sb2z"
server_key: "d7934c5cc7005f759f9b3938ebcb241f"
server_key_iv: "41a74c875b91074d4316de114ee11915"
server_key_version: "encryptionkey@0"
I20260812 17:52:23.948503 8622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:23.967713 8622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:23.971935 8622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:23.972235 8622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:23.972985 8622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:23.973932 8622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:23.973984 8622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:23.974032 8622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:23.974049 8622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:23.989121 8622 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:36073
I20260812 17:52:23.989110 8815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:36073 every 8 connection(s)
I20260812 17:52:23.990540 8622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:52:23.992964 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 8622
I20260812 17:52:23.993157 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:52:23.993594 13464 external_mini_cluster.cc:1428] Setting key fdb96676ed2a755fb5b11312c1e10e35
Aug 12 17:52:24 dist-test-slave-sb2z krb5kdc[8053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557143, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:52:24.010499 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:23.993285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:41853 (local address 127.13.38.62:40115)
0812 17:52:23.993579 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:23.993586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:23.996003 (+ 2417us) server_negotiation.cc:407] Connection header received
0812 17:52:23.997217 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:23.997220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:23.997278 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:23.997359 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:23.998786 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:23.999365 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.000929 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.001170 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.001296 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.004519 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:24.004523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:24.004525 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:24.004560 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:24.006382 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:24.007707 (+ 1325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:24.007711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:24.007713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:24.007829 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:24.008642 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:24.008646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:24.008648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:24.008961 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:24.009100 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:24.010177 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0812 17:52:24.010358 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260812 17:52:24.011365 8820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:23.994135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:40115 (local address 127.13.38.3:41853)
0812 17:52:23.995810 (+ 1675us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:23.995884 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:23.996989 (+ 1105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:23.997559 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:23.997568 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:23.997983 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:23.998620 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:23.998638 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:23.999559 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:23.999564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.000739 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.000753 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.001359 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.001363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.001587 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.002350 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:52:24.002370 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:24.004242 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:24.006658 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:24.006670 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:24.006706 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:24.007498 (+ 792us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:24.008009 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:24.008015 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:24.008017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:24.008420 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:24.009203 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:24.009213 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:24.009685 (+ 472us) client_negotiation.cc:769] Sending connection context
0812 17:52:24.009937 (+ 252us) client_negotiation.cc:240] Negotiation successful
0812 17:52:24.010371 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303,"thread_start_us":804,"threads_started":1}
I20260812 17:52:24.013195 8817 heartbeater.cc:347] Connected to a master server at 127.13.38.62:40115
I20260812 17:52:24.013537 8817 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:24.014189 8817 heartbeater.cc:511] Master 127.13.38.62:40115 requested a full tablet report, sending...
I20260812 17:52:24.015792 8129 ts_manager.cc:194] Registered new tserver with Master: d38a9d35e943444a94a31ed66a837efb (127.13.38.3:36073)
I20260812 17:52:24.016487 8129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:41853
I20260812 17:52:24.023059 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:52:24.034826 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.025686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47768 (local address 127.13.38.62:40115)
0812 17:52:24.026050 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:24.026056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:24.026301 (+ 245us) server_negotiation.cc:407] Connection header received
0812 17:52:24.026478 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:24.026482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:24.026535 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:24.026622 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:24.027627 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.028211 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.029136 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.029358 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.029452 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.030938 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:24.030941 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:24.030943 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:24.030973 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:24.032652 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:24.033221 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:24.033224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:24.033225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:24.033339 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:24.033697 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:24.033700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:24.033701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:24.033980 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:24.034096 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:24.034443 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 17:52:24.034611 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260812 17:52:24.039028 8129 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:52:24.045408 8129 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:52:24.065038 8829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.056775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:40893 (local address 127.0.0.1:50786)
0812 17:52:24.057753 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:24.057768 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:24.057920 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:24.060455 (+ 2535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:24.060458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:24.060476 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:24.060576 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.060586 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.061947 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.061950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.063126 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.063138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.064331 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.064335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.064472 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.064523 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 17:52:24.064674 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 17:52:24.064837 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":56,"thread_start_us":117,"threads_started":1}
I20260812 17:52:24.065363 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.056509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:36073 (local address 127.0.0.1:56852)
0812 17:52:24.057402 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:24.057540 (+ 138us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:24.058031 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:24.060118 (+ 2087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:24.060121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:24.060141 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:24.060281 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.060288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.061906 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.061910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.063673 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.063689 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.064932 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.064936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.065032 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.065062 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 17:52:24.065142 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 17:52:24.065240 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":331,"thread_start_us":116,"threads_started":1}
I20260812 17:52:24.065801 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.057612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.13.38.2:42177)
0812 17:52:24.059231 (+ 1619us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:24.059248 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:24.059286 (+ 38us) server_negotiation.cc:407] Connection header received
0812 17:52:24.059383 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:24.059390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:24.059709 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:24.059902 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:24.060289 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.061692 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.063821 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.064700 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.064815 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.064963 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:24.065446 (+ 483us) server_negotiation.cc:299] Negotiation successful
0812 17:52:24.065600 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1415,"thread_start_us":118,"threads_started":1}
I20260812 17:52:24.065851 8836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.057960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:42177 (local address 127.0.0.1:43290)
0812 17:52:24.058699 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:24.058719 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:24.058841 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:24.059918 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:24.059926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:24.059956 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:24.060136 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.060144 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.061906 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.061910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.063686 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.063693 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.065040 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.065044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.065146 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.065171 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 17:52:24.065242 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 17:52:24.065324 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":197,"thread_start_us":101,"threads_started":1}
I20260812 17:52:24.066115 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.057711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50786 (local address 127.13.38.1:40893)
0812 17:52:24.059849 (+ 2138us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:24.059858 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:24.059883 (+ 25us) server_negotiation.cc:407] Connection header received
0812 17:52:24.060039 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:24.060044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:24.060258 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:24.060443 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:24.060695 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.061807 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.063272 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.064170 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.064294 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.064465 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:24.065794 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0812 17:52:24.065958 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2011,"thread_start_us":113,"threads_started":1}
I20260812 17:52:24.066437 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.057546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56852 (local address 127.13.38.3:36073)
0812 17:52:24.059424 (+ 1878us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:24.059445 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:24.059467 (+ 22us) server_negotiation.cc:407] Connection header received
0812 17:52:24.059558 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:24.059563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:24.059725 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:24.059901 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:24.060487 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.061756 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.063861 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.064609 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.064716 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.064862 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:24.066108 (+ 1246us) server_negotiation.cc:299] Negotiation successful
0812 17:52:24.066263 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1676,"thread_start_us":111,"threads_started":1}
I20260812 17:52:24.069468 8325 tablet_service.cc:1467] Processing CreateTablet for tablet 6953b3d67ec34eaeb4ce8e7b917c31b9 (DEFAULT_TABLE table=test-table [id=dc221112472f432da0460c43ca373f4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:24.069461 8716 tablet_service.cc:1467] Processing CreateTablet for tablet 6953b3d67ec34eaeb4ce8e7b917c31b9 (DEFAULT_TABLE table=test-table [id=dc221112472f432da0460c43ca373f4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:24.069461 8514 tablet_service.cc:1467] Processing CreateTablet for tablet 6953b3d67ec34eaeb4ce8e7b917c31b9 (DEFAULT_TABLE table=test-table [id=dc221112472f432da0460c43ca373f4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:24.070770 8325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6953b3d67ec34eaeb4ce8e7b917c31b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:24.070930 8514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6953b3d67ec34eaeb4ce8e7b917c31b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:24.071008 8716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6953b3d67ec34eaeb4ce8e7b917c31b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:24.078281 8837 tablet_bootstrap.cc:492] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425: Bootstrap starting.
I20260812 17:52:24.080165 8838 tablet_bootstrap.cc:492] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1: Bootstrap starting.
I20260812 17:52:24.082268 8837 tablet_bootstrap.cc:654] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:24.082862 8838 tablet_bootstrap.cc:654] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:24.083442 8837 log.cc:824] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:24.084340 8838 log.cc:824] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:24.084504 8839 tablet_bootstrap.cc:492] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb: Bootstrap starting.
I20260812 17:52:24.086359 8837 tablet_bootstrap.cc:492] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425: No bootstrap required, opened a new log
I20260812 17:52:24.086621 8837 ts_tablet_manager.cc:1397] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260812 17:52:24.086784 8838 tablet_bootstrap.cc:492] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1: No bootstrap required, opened a new log
I20260812 17:52:24.087002 8838 ts_tablet_manager.cc:1397] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 17:52:24.087450 8839 tablet_bootstrap.cc:654] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:24.089174 8839 log.cc:824] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:24.091709 8838 raft_consensus.cc:348] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d38a9d35e943444a94a31ed66a837efb" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36073 } } peers { permanent_uuid: "b2f5acb303ba41e5b0842e5a77bc2425" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 40893 } } peers { permanent_uuid: "083c8ca907e54287a386fc144e800ec1" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 42177 } }
I20260812 17:52:24.092247 8838 raft_consensus.cc:374] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:24.092362 8838 raft_consensus.cc:729] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083c8ca907e54287a386fc144e800ec1, State: Initialized, Role: FOLLOWER
I20260812 17:52:24.092694 8839 tablet_bootstrap.cc:492] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb: No bootstrap required, opened a new log
I20260812 17:52:24.093046 8839 ts_tablet_manager.cc:1397] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 17:52:24.093209 8838 consensus_queue.cc:260] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d38a9d35e943444a94a31ed66a837efb" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36073 } } peers { permanent_uuid: "b2f5acb303ba41e5b0842e5a77bc2425" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 40893 } } peers { permanent_uuid: "083c8ca907e54287a386fc144e800ec1" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 42177 } }
I20260812 17:52:24.094650 8838 ts_tablet_manager.cc:1428] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260812 17:52:24.095132 8616 heartbeater.cc:503] Master 127.13.38.62:40115 was elected leader, sending a full tablet report...
I20260812 17:52:24.095212 8837 raft_consensus.cc:348] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d38a9d35e943444a94a31ed66a837efb" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36073 } } peers { permanent_uuid: "b2f5acb303ba41e5b0842e5a77bc2425" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 40893 } } peers { permanent_uuid: "083c8ca907e54287a386fc144e800ec1" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 42177 } }
I20260812 17:52:24.095569 8837 raft_consensus.cc:374] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:24.095667 8837 raft_consensus.cc:729] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2f5acb303ba41e5b0842e5a77bc2425, State: Initialized, Role: FOLLOWER
I20260812 17:52:24.096259 8837 consensus_queue.cc:260] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d38a9d35e943444a94a31ed66a837efb" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36073 } } peers { permanent_uuid: "b2f5acb303ba41e5b0842e5a77bc2425" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 40893 } } peers { permanent_uuid: "083c8ca907e54287a386fc144e800ec1" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 42177 } }
I20260812 17:52:24.097380 8839 raft_consensus.cc:348] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d38a9d35e943444a94a31ed66a837efb" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36073 } } peers { permanent_uuid: "b2f5acb303ba41e5b0842e5a77bc2425" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 40893 } } peers { permanent_uuid: "083c8ca907e54287a386fc144e800ec1" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 42177 } }
I20260812 17:52:24.097762 8839 raft_consensus.cc:374] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:24.097874 8839 raft_consensus.cc:729] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d38a9d35e943444a94a31ed66a837efb, State: Initialized, Role: FOLLOWER
I20260812 17:52:24.098452 8839 consensus_queue.cc:260] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d38a9d35e943444a94a31ed66a837efb" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36073 } } peers { permanent_uuid: "b2f5acb303ba41e5b0842e5a77bc2425" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 40893 } } peers { permanent_uuid: "083c8ca907e54287a386fc144e800ec1" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 42177 } }
I20260812 17:52:24.098735 8837 ts_tablet_manager.cc:1428] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425: Time spent starting tablet: real 0.012s user 0.009s sys 0.000s
I20260812 17:52:24.099234 8426 heartbeater.cc:503] Master 127.13.38.62:40115 was elected leader, sending a full tablet report...
I20260812 17:52:24.099658 8839 ts_tablet_manager.cc:1428] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 17:52:24.100020 8817 heartbeater.cc:503] Master 127.13.38.62:40115 was elected leader, sending a full tablet report...
I20260812 17:52:24.153452 8845 raft_consensus.cc:482] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:52:24.153760 8845 raft_consensus.cc:504] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d38a9d35e943444a94a31ed66a837efb" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36073 } } peers { permanent_uuid: "b2f5acb303ba41e5b0842e5a77bc2425" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 40893 } } peers { permanent_uuid: "083c8ca907e54287a386fc144e800ec1" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 42177 } }
I20260812 17:52:24.155092 8845 leader_election.cc:288] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d38a9d35e943444a94a31ed66a837efb (127.13.38.3:36073), b2f5acb303ba41e5b0842e5a77bc2425 (127.13.38.1:40893)
I20260812 17:52:24.160066 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.155647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:47161 (local address 127.13.38.1:40893)
0812 17:52:24.155819 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:24.155824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:24.156220 (+ 396us) server_negotiation.cc:407] Connection header received
0812 17:52:24.156279 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:24.156282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:24.156332 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:24.156425 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:24.156770 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.157471 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.158744 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.159255 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.159367 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.159423 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:24.159828 (+ 405us) server_negotiation.cc:299] Negotiation successful
0812 17:52:24.159919 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 17:52:24.160099 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.155542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:57533 (local address 127.13.38.3:36073)
0812 17:52:24.155702 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:24.155709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:24.155897 (+ 188us) server_negotiation.cc:407] Connection header received
0812 17:52:24.156055 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:24.156061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:24.156124 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:24.156204 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:24.156563 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.157330 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.158624 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.159153 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.159309 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.159407 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:24.159828 (+ 421us) server_negotiation.cc:299] Negotiation successful
0812 17:52:24.159919 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 17:52:24.160086 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.155429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:36073 (local address 127.13.38.2:57533)
0812 17:52:24.155799 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:24.155817 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:24.155939 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:24.156243 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:24.156246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:24.156269 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:24.156418 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.156427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.157458 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.157462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.158462 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.158475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.159460 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.159464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.159621 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.159681 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 17:52:24.159771 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 17:52:24.159911 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":86}
I20260812 17:52:24.160094 8850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.155432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:40893 (local address 127.13.38.2:47161)
0812 17:52:24.155959 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:24.155995 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:24.156114 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:24.156450 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:24.156453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:24.156478 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:24.156618 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.156626 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.157677 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.157682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.158585 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.158600 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.159408 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.159412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.159569 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.159684 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 17:52:24.159771 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 17:52:24.159935 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":252,"thread_start_us":135,"threads_started":1}
I20260812 17:52:24.161105 8352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6953b3d67ec34eaeb4ce8e7b917c31b9" candidate_uuid: "083c8ca907e54287a386fc144e800ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2f5acb303ba41e5b0842e5a77bc2425" is_pre_election: true
I20260812 17:52:24.161103 8746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6953b3d67ec34eaeb4ce8e7b917c31b9" candidate_uuid: "083c8ca907e54287a386fc144e800ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d38a9d35e943444a94a31ed66a837efb" is_pre_election: true
I20260812 17:52:24.161512 8352 raft_consensus.cc:2454] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 083c8ca907e54287a386fc144e800ec1 in term 0.
I20260812 17:52:24.161512 8746 raft_consensus.cc:2454] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 083c8ca907e54287a386fc144e800ec1 in term 0.
I20260812 17:52:24.162139 8456 leader_election.cc:302] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 083c8ca907e54287a386fc144e800ec1, b2f5acb303ba41e5b0842e5a77bc2425, d38a9d35e943444a94a31ed66a837efb; no voters:
I20260812 17:52:24.162547 8845 raft_consensus.cc:2781] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:52:24.162673 8845 raft_consensus.cc:482] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:52:24.162740 8845 raft_consensus.cc:3037] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:24.164008 8845 raft_consensus.cc:504] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d38a9d35e943444a94a31ed66a837efb" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36073 } } peers { permanent_uuid: "b2f5acb303ba41e5b0842e5a77bc2425" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 40893 } } peers { permanent_uuid: "083c8ca907e54287a386fc144e800ec1" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 42177 } }
I20260812 17:52:24.164422 8845 leader_election.cc:288] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [CANDIDATE]: Term 1 election: Requested vote from peers d38a9d35e943444a94a31ed66a837efb (127.13.38.3:36073), b2f5acb303ba41e5b0842e5a77bc2425 (127.13.38.1:40893)
I20260812 17:52:24.164973 8352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6953b3d67ec34eaeb4ce8e7b917c31b9" candidate_uuid: "083c8ca907e54287a386fc144e800ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2f5acb303ba41e5b0842e5a77bc2425"
I20260812 17:52:24.164973 8746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6953b3d67ec34eaeb4ce8e7b917c31b9" candidate_uuid: "083c8ca907e54287a386fc144e800ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d38a9d35e943444a94a31ed66a837efb"
I20260812 17:52:24.165134 8352 raft_consensus.cc:3037] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:24.165135 8746 raft_consensus.cc:3037] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:24.166600 8352 raft_consensus.cc:2454] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 083c8ca907e54287a386fc144e800ec1 in term 1.
I20260812 17:52:24.166600 8746 raft_consensus.cc:2454] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 083c8ca907e54287a386fc144e800ec1 in term 1.
I20260812 17:52:24.167035 8456 leader_election.cc:302] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 083c8ca907e54287a386fc144e800ec1, b2f5acb303ba41e5b0842e5a77bc2425; no voters:
I20260812 17:52:24.167285 8845 raft_consensus.cc:2781] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:24.167739 8845 raft_consensus.cc:686] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 1 LEADER]: Becoming Leader. State: Replica: 083c8ca907e54287a386fc144e800ec1, State: Running, Role: LEADER
I20260812 17:52:24.168185 8845 consensus_queue.cc:237] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d38a9d35e943444a94a31ed66a837efb" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36073 } } peers { permanent_uuid: "b2f5acb303ba41e5b0842e5a77bc2425" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 40893 } } peers { permanent_uuid: "083c8ca907e54287a386fc144e800ec1" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 42177 } }
I20260812 17:52:24.172252 8126 catalog_manager.cc:5495] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 083c8ca907e54287a386fc144e800ec1 (127.13.38.2). New cstate: current_term: 1 leader_uuid: "083c8ca907e54287a386fc144e800ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d38a9d35e943444a94a31ed66a837efb" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2f5acb303ba41e5b0842e5a77bc2425" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 40893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "083c8ca907e54287a386fc144e800ec1" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 42177 } health_report { overall_health: HEALTHY } } }
I20260812 17:52:24.192142 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.187640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.13.38.2:42177)
0812 17:52:24.187776 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:24.187780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:24.187846 (+ 66us) server_negotiation.cc:407] Connection header received
0812 17:52:24.188158 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:24.188161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:24.188238 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:24.188349 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:52:24.188674 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.189214 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.190251 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.190510 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.190626 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.191036 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:52:24.191676 (+ 640us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:52:24.191789 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:24.191918 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 17:52:24.191989 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 17:52:24.199147 8746 raft_consensus.cc:1261] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb [term 1 FOLLOWER]: Refusing update from remote peer 083c8ca907e54287a386fc144e800ec1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:52:24.199147 8352 raft_consensus.cc:1261] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425 [term 1 FOLLOWER]: Refusing update from remote peer 083c8ca907e54287a386fc144e800ec1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:52:24.200103 8845 consensus_queue.cc:1035] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2f5acb303ba41e5b0842e5a77bc2425" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 40893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:52:24.200454 8852 consensus_queue.cc:1035] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d38a9d35e943444a94a31ed66a837efb" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 36073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 17:52:24.203070 8427 tablet.cc:2369] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:52:24.212639 8859 mvcc.cc:205] Tried to move back new op lower bound from 7317738062630330368 to 7317738062515990528. Current Snapshot: MvccSnapshot[applied={T|T < 7317738062630330368}]
I20260812 17:52:24.221873 8127 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:52:24.222060 8127 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:52:24.225155 8127 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d78d9b5b228e4bbea8d21ce80561d2a5: Sending DeleteTablet for 3 replicas of tablet 6953b3d67ec34eaeb4ce8e7b917c31b9
I20260812 17:52:24.226200 8325 tablet_service.cc:1514] Processing DeleteTablet for tablet 6953b3d67ec34eaeb4ce8e7b917c31b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:24 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:50786
I20260812 17:52:24.226200 8514 tablet_service.cc:1514] Processing DeleteTablet for tablet 6953b3d67ec34eaeb4ce8e7b917c31b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:24 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:43290
I20260812 17:52:24.226847 8874 tablet_replica.cc:330] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1: stopping tablet replica
I20260812 17:52:24.226845 8716 tablet_service.cc:1514] Processing DeleteTablet for tablet 6953b3d67ec34eaeb4ce8e7b917c31b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:24 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:56852
I20260812 17:52:24.227218 8873 tablet_replica.cc:330] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425: stopping tablet replica
I20260812 17:52:24.227352 8875 tablet_replica.cc:330] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb: stopping tablet replica
I20260812 17:52:24.227664 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 8220
I20260812 17:52:24.227797 8875 raft_consensus.cc:2229] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:52:24.227986 8873 raft_consensus.cc:2229] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P b2f5acb303ba41e5b0842e5a77bc2425 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:52:24.228176 8874 raft_consensus.cc:2229] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:52:24.228214 8875 raft_consensus.cc:2258] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:52:24.228636 8874 raft_consensus.cc:2258] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:52:24.232455 8874 ts_tablet_manager.cc:1905] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:52:24.232597 8875 ts_tablet_manager.cc:1905] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:52:24.236720 8875 ts_tablet_manager.cc:1918] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:52:24.236896 8875 log.cc:1196] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/6953b3d67ec34eaeb4ce8e7b917c31b9
I20260812 17:52:24.237145 8874 ts_tablet_manager.cc:1918] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:52:24.237334 8875 ts_tablet_manager.cc:1939] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P d38a9d35e943444a94a31ed66a837efb: Deleting consensus metadata
I20260812 17:52:24.237555 8874 log.cc:1196] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/6953b3d67ec34eaeb4ce8e7b917c31b9
I20260812 17:52:24.238046 8874 ts_tablet_manager.cc:1939] T 6953b3d67ec34eaeb4ce8e7b917c31b9 P 083c8ca907e54287a386fc144e800ec1: Deleting consensus metadata
I20260812 17:52:24.239214 8104 catalog_manager.cc:4841] TS d38a9d35e943444a94a31ed66a837efb (127.13.38.3:36073): tablet 6953b3d67ec34eaeb4ce8e7b917c31b9 (table test-table [id=dc221112472f432da0460c43ca373f4c]) successfully deleted
I20260812 17:52:24.239508 8104 catalog_manager.cc:4841] TS 083c8ca907e54287a386fc144e800ec1 (127.13.38.2:42177): tablet 6953b3d67ec34eaeb4ce8e7b917c31b9 (table test-table [id=dc221112472f432da0460c43ca373f4c]) successfully deleted
W20260812 17:52:24.247630 8101 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:40893 (error 108)
I20260812 17:52:24.247723 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 8431
I20260812 17:52:24.251842 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.251393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:40893 (local address 127.0.0.1:50802)
0812 17:52:24.251642 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:24.251724 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:40893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260812 17:52:24.252063 8101 catalog_manager.cc:4584] TS b2f5acb303ba41e5b0842e5a77bc2425 (127.13.38.1:40893): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6953b3d67ec34eaeb4ce8e7b917c31b9: Network error: Client connection negotiation failed: client connection to 127.13.38.1:40893: connect: Connection refused (error 111)
I20260812 17:52:24.267419 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 8622
I20260812 17:52:24.283236 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 8070
2026-08-12T17:52:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2903 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T17:52:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:24Z Disabled control of system clock
I20260812 17:52:24.328908 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:37263
--webserver_interface=127.13.38.62
--webserver_port=0
--builtin_ntp_servers=127.13.38.20:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:37263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 17:52:24.451825 8883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:24.452149 8883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:24.452188 8883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:24.456561 8883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:24.456678 8883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:24.456698 8883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:24.456717 8883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:24.456733 8883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:24.462669 8883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:37263
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:37263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:24.463958 8883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:24.465461 8883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:24.475088 8891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:24.475274 8883 server_base.cc:1034] running on GCE node
W20260812 17:52:24.475081 8894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:24.475137 8889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:24.476161 8883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:24.477355 8883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:24.478626 8883 hybrid_clock.cc:648] HybridClock initialized: now 1786557144478587 us; error 57 us; skew 500 ppm
I20260812 17:52:24.481038 8883 webserver.cc:460] Webserver started at http://127.13.38.62:42553/ using document root <none> and password file <none>
I20260812 17:52:24.481730 8883 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:24.481835 8883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:24.482097 8883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:24.484519 8883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "f94319a79d7f4a8ab339e5959d082e6f"
format_stamp: "Formatted at 2026-08-12 17:52:24 on dist-test-slave-sb2z"
server_key: "b9cabdd9f6a3a320d022bc16bbfa77d4"
server_key_iv: "04d4d2f4eb4235d94d77230a70a46540"
server_key_version: "encryptionkey@0"
I20260812 17:52:24.485196 8883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "f94319a79d7f4a8ab339e5959d082e6f"
format_stamp: "Formatted at 2026-08-12 17:52:24 on dist-test-slave-sb2z"
server_key: "b9cabdd9f6a3a320d022bc16bbfa77d4"
server_key_iv: "04d4d2f4eb4235d94d77230a70a46540"
server_key_version: "encryptionkey@0"
I20260812 17:52:24.489562 8883 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:52:24.492623 8904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:24.494103 8883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 17:52:24.494288 8883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "f94319a79d7f4a8ab339e5959d082e6f"
format_stamp: "Formatted at 2026-08-12 17:52:24 on dist-test-slave-sb2z"
server_key: "b9cabdd9f6a3a320d022bc16bbfa77d4"
server_key_iv: "04d4d2f4eb4235d94d77230a70a46540"
server_key_version: "encryptionkey@0"
I20260812 17:52:24.494428 8883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:24.510142 8883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:24.511004 8883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:24.511257 8883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:24.519686 8883 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:37263
I20260812 17:52:24.519699 8991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:37263 every 8 connection(s)
I20260812 17:52:24.520845 8883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:24.524501 8993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:24.527050 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 8883
I20260812 17:52:24.527171 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:24.527470 13464 external_mini_cluster.cc:1428] Setting key 93e097f3dc89890afa08963c91d05dfe
I20260812 17:52:24.531155 8993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f: Bootstrap starting.
I20260812 17:52:24.533694 8993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:24.534526 8993 log.cc:824] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:24.537165 8993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f: No bootstrap required, opened a new log
I20260812 17:52:24.538089 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.529162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39068 (local address 127.13.38.62:37263)
0812 17:52:24.529983 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:24.530001 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:24.530044 (+ 43us) server_negotiation.cc:407] Connection header received
0812 17:52:24.530755 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:24.530776 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:24.531223 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:24.531675 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:24.532165 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.533187 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.534190 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.534706 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.534888 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.535866 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:24.535871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:24.535887 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:24.536336 (+ 449us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:24.536400 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:24.536430 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:24.536659 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:24.536824 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 17:52:24.537179 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":175,"threads_started":1}
I20260812 17:52:24.541828 8993 raft_consensus.cc:348] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94319a79d7f4a8ab339e5959d082e6f" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37263 } }
I20260812 17:52:24.542120 8993 raft_consensus.cc:374] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:24.542182 8993 raft_consensus.cc:729] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f94319a79d7f4a8ab339e5959d082e6f, State: Initialized, Role: FOLLOWER
I20260812 17:52:24.542629 8993 consensus_queue.cc:260] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94319a79d7f4a8ab339e5959d082e6f" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37263 } }
I20260812 17:52:24.542778 8993 raft_consensus.cc:388] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:24.542822 8993 raft_consensus.cc:482] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:24.542903 8993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:24.544271 8993 raft_consensus.cc:504] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94319a79d7f4a8ab339e5959d082e6f" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37263 } }
I20260812 17:52:24.544664 8993 leader_election.cc:302] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f94319a79d7f4a8ab339e5959d082e6f; no voters:
I20260812 17:52:24.544975 8993 leader_election.cc:288] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:24.545130 9003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:24.545392 9003 raft_consensus.cc:686] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [term 1 LEADER]: Becoming Leader. State: Replica: f94319a79d7f4a8ab339e5959d082e6f, State: Running, Role: LEADER
I20260812 17:52:24.545800 9003 consensus_queue.cc:237] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94319a79d7f4a8ab339e5959d082e6f" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37263 } }
I20260812 17:52:24.546422 8993 sys_catalog.cc:564] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:24.547745 9005 sys_catalog.cc:455] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f94319a79d7f4a8ab339e5959d082e6f. Latest consensus state: current_term: 1 leader_uuid: "f94319a79d7f4a8ab339e5959d082e6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94319a79d7f4a8ab339e5959d082e6f" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37263 } } }
I20260812 17:52:24.547914 9005 sys_catalog.cc:458] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:24.548277 9004 sys_catalog.cc:455] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f94319a79d7f4a8ab339e5959d082e6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94319a79d7f4a8ab339e5959d082e6f" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 37263 } } }
I20260812 17:52:24.548403 9004 sys_catalog.cc:458] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:24.548354 9016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:24.552572 9016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:24.559520 9016 catalog_manager.cc:1347] Generated new cluster ID: 3392f257044848aa9ff8d2163f2894f2
I20260812 17:52:24.559700 9016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:24.572234 9016 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:24.573355 9016 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:24.590710 9016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f: Generated new TSK 0
I20260812 17:52:24.592466 9016 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:24.598260 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:37263
--builtin_ntp_servers=127.13.38.20:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 17:52:24.735802 9032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:24.736147 9032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:24.736191 9032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:24.741003 9032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:24.741117 9032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:24.741273 9032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:52:24.747126 9032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:37263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:24.748490 9032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:24.750011 9032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:24.760845 9044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:24.760798 9041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:24.761137 9032 server_base.cc:1034] running on GCE node
W20260812 17:52:24.760833 9039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:24.761762 9032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:24.762586 9032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:24.763769 9032 hybrid_clock.cc:648] HybridClock initialized: now 1786557144763722 us; error 62 us; skew 500 ppm
I20260812 17:52:24.766036 9032 webserver.cc:460] Webserver started at http://127.13.38.1:36687/ using document root <none> and password file <none>
I20260812 17:52:24.766690 9032 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:24.766763 9032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:24.766961 9032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:24.769408 9032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c"
format_stamp: "Formatted at 2026-08-12 17:52:24 on dist-test-slave-sb2z"
server_key: "0c31826fe4c9f91f2cbfd8c29d4e4653"
server_key_iv: "ea4849690cbbea38d61ecd63d72c194c"
server_key_version: "encryptionkey@0"
I20260812 17:52:24.770037 9032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c"
format_stamp: "Formatted at 2026-08-12 17:52:24 on dist-test-slave-sb2z"
server_key: "0c31826fe4c9f91f2cbfd8c29d4e4653"
server_key_iv: "ea4849690cbbea38d61ecd63d72c194c"
server_key_version: "encryptionkey@0"
I20260812 17:52:24.774545 9032 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 17:52:24.777812 9055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:24.779405 9032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:52:24.779620 9032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c"
format_stamp: "Formatted at 2026-08-12 17:52:24 on dist-test-slave-sb2z"
server_key: "0c31826fe4c9f91f2cbfd8c29d4e4653"
server_key_iv: "ea4849690cbbea38d61ecd63d72c194c"
server_key_version: "encryptionkey@0"
I20260812 17:52:24.779783 9032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:24.830111 9032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:24.831007 9032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:24.831243 9032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:24.832059 9032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:24.833230 9032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:24.833288 9032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:24.833367 9032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:24.833402 9032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:24.844456 9032 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:37213
I20260812 17:52:24.844466 9230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:37213 every 8 connection(s)
I20260812 17:52:24.845597 9032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:52:24.849318 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 9032
I20260812 17:52:24.849462 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:52:24.849794 13464 external_mini_cluster.cc:1428] Setting key 261ba845cee3d3350695f2e8b7646c79
I20260812 17:52:24.852099 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:37263
--builtin_ntp_servers=127.13.38.20:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 17:52:24.857544 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.848223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:39431 (local address 127.13.38.62:37263)
0812 17:52:24.848471 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:24.848475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:24.849732 (+ 1257us) server_negotiation.cc:407] Connection header received
0812 17:52:24.851493 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:24.851496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:24.851579 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:24.851687 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:24.852755 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.853291 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.855112 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.855381 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.855500 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.856973 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:24.856976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:24.856978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:24.856993 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:24.857014 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:24.857023 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:24.857135 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:24.857346 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 17:52:24.857406 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 17:52:24.859038 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:24.848678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:37263 (local address 127.13.38.1:39431)
0812 17:52:24.849544 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:24.849611 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:24.851259 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:24.851889 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:24.851896 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:24.851998 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:24.852606 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.852620 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.853436 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.853439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.854890 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:24.854906 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:24.855601 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:24.855608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:24.855864 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:24.856612 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:52:24.856616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:24.856677 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:52:24.856776 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:24.857151 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:24.857166 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:24.857177 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 17:52:24.857347 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 17:52:24.857751 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":152,"threads_started":1}
I20260812 17:52:24.860756 9231 heartbeater.cc:347] Connected to a master server at 127.13.38.62:37263
I20260812 17:52:24.861094 9231 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:24.861745 9231 heartbeater.cc:511] Master 127.13.38.62:37263 requested a full tablet report, sending...
I20260812 17:52:24.863642 8921 ts_manager.cc:194] Registered new tserver with Master: 94f6ec8a30ca41d59f49cf2b8075ec6c (127.13.38.1:37213)
I20260812 17:52:24.864943 8921 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.38.1:39431
W20260812 17:52:24.993268 9235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:24.993654 9235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:24.993782 9235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:24.998733 9235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:24.998819 9235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:24.998919 9235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:52:25.005235 9235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:37263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:25.006629 9235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:25.008343 9235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:25.020655 9244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:25.020663 9245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:25.020660 9247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:25.021211 9235 server_base.cc:1034] running on GCE node
I20260812 17:52:25.021824 9235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:25.022715 9235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:25.023933 9235 hybrid_clock.cc:648] HybridClock initialized: now 1786557145023902 us; error 42 us; skew 500 ppm
I20260812 17:52:25.026322 9235 webserver.cc:460] Webserver started at http://127.13.38.2:40371/ using document root <none> and password file <none>
I20260812 17:52:25.027004 9235 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:25.027120 9235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:25.027386 9235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:25.029778 9235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "9ac7f566bbaa4ba98329796135a9cab4"
format_stamp: "Formatted at 2026-08-12 17:52:25 on dist-test-slave-sb2z"
server_key: "d05e213625b44731d20ce88e0f0c5717"
server_key_iv: "2adde4f92c7547d2a2d0904bd649c63b"
server_key_version: "encryptionkey@0"
I20260812 17:52:25.030447 9235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "9ac7f566bbaa4ba98329796135a9cab4"
format_stamp: "Formatted at 2026-08-12 17:52:25 on dist-test-slave-sb2z"
server_key: "d05e213625b44731d20ce88e0f0c5717"
server_key_iv: "2adde4f92c7547d2a2d0904bd649c63b"
server_key_version: "encryptionkey@0"
I20260812 17:52:25.034839 9235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 17:52:25.038137 9258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:25.039777 9235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:25.039944 9235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "9ac7f566bbaa4ba98329796135a9cab4"
format_stamp: "Formatted at 2026-08-12 17:52:25 on dist-test-slave-sb2z"
server_key: "d05e213625b44731d20ce88e0f0c5717"
server_key_iv: "2adde4f92c7547d2a2d0904bd649c63b"
server_key_version: "encryptionkey@0"
I20260812 17:52:25.040105 9235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:25.066311 9235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:25.067241 9235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:25.067498 9235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:25.068267 9235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:25.069478 9235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:25.069563 9235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:25.069649 9235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:25.069696 9235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:25.080499 9235 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:43509
I20260812 17:52:25.080520 9421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:43509 every 8 connection(s)
I20260812 17:52:25.081818 9235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:52:25.083014 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 9235
I20260812 17:52:25.083124 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:52:25.083438 13464 external_mini_cluster.cc:1428] Setting key fa740b1c0f9e6d1bf826c2a425267d3d
I20260812 17:52:25.086064 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:37263
--builtin_ntp_servers=127.13.38.20:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 17:52:25.096395 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.085042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:42485 (local address 127.13.38.62:37263)
0812 17:52:25.085272 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:25.085277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:25.086768 (+ 1491us) server_negotiation.cc:407] Connection header received
0812 17:52:25.088753 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:25.088756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:25.088814 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:25.088910 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:25.090618 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.091374 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.093726 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.093976 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.094094 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.095731 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:25.095734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:25.095736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:25.095753 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:25.095776 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:25.095786 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:25.095876 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:25.096189 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 17:52:25.096263 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 17:52:25.097790 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.085672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:37263 (local address 127.13.38.2:42485)
0812 17:52:25.086576 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:25.086638 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:25.088420 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:25.089172 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:25.089185 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:25.089331 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:25.090348 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.090376 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.091717 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.091724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.093530 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.093549 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.094185 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.094192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.094476 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.095300 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:52:25.095304 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:25.095380 (+ 76us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:52:25.095496 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:25.095909 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:25.095923 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:25.095933 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 17:52:25.096097 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 17:52:25.096508 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":241,"threads_started":1}
I20260812 17:52:25.099169 9423 heartbeater.cc:347] Connected to a master server at 127.13.38.62:37263
I20260812 17:52:25.099505 9423 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:25.100414 9423 heartbeater.cc:511] Master 127.13.38.62:37263 requested a full tablet report, sending...
I20260812 17:52:25.101838 8921 ts_manager.cc:194] Registered new tserver with Master: 9ac7f566bbaa4ba98329796135a9cab4 (127.13.38.2:43509)
I20260812 17:52:25.102520 8921 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.38.2:42485
W20260812 17:52:25.221825 9427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:25.222220 9427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:25.222267 9427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:25.226989 9427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:25.227075 9427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:25.227293 9427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:52:25.234159 9427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:37263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:25.235680 9427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:25.237716 9427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:25.249806 9435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:25.250108 9427 server_base.cc:1034] running on GCE node
W20260812 17:52:25.249799 9434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:25.249851 9437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:25.250982 9427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:25.251910 9427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:25.253132 9427 hybrid_clock.cc:648] HybridClock initialized: now 1786557145253116 us; error 38 us; skew 500 ppm
I20260812 17:52:25.255745 9427 webserver.cc:460] Webserver started at http://127.13.38.3:45033/ using document root <none> and password file <none>
I20260812 17:52:25.256466 9427 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:25.256578 9427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:25.256929 9427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:25.259779 9427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "f399674d43804bb5a1bfdb0e153e5881"
format_stamp: "Formatted at 2026-08-12 17:52:25 on dist-test-slave-sb2z"
server_key: "aa5c597f7568d863d6d2422647e9a407"
server_key_iv: "c1809cecbdd0ec46538b33246e6e3f05"
server_key_version: "encryptionkey@0"
I20260812 17:52:25.260546 9427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "f399674d43804bb5a1bfdb0e153e5881"
format_stamp: "Formatted at 2026-08-12 17:52:25 on dist-test-slave-sb2z"
server_key: "aa5c597f7568d863d6d2422647e9a407"
server_key_iv: "c1809cecbdd0ec46538b33246e6e3f05"
server_key_version: "encryptionkey@0"
I20260812 17:52:25.265549 9427 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 17:52:25.269382 9448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:25.271068 9427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 17:52:25.271255 9427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "f399674d43804bb5a1bfdb0e153e5881"
format_stamp: "Formatted at 2026-08-12 17:52:25 on dist-test-slave-sb2z"
server_key: "aa5c597f7568d863d6d2422647e9a407"
server_key_iv: "c1809cecbdd0ec46538b33246e6e3f05"
server_key_version: "encryptionkey@0"
I20260812 17:52:25.271404 9427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:25.288725 9427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:25.289573 9427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:25.289803 9427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:25.290874 9427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:25.292105 9427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:25.292182 9427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:25.292254 9427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:25.292295 9427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:25.302907 9427 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:43673
I20260812 17:52:25.302935 9623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:43673 every 8 connection(s)
I20260812 17:52:25.304013 9427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:52:25.307843 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 9427
I20260812 17:52:25.308007 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:52:25.308348 13464 external_mini_cluster.cc:1428] Setting key 807673555f42f249fcf8680c6dc38e2d
I20260812 17:52:25.315928 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.306221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:39871 (local address 127.13.38.62:37263)
0812 17:52:25.306495 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:25.306500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:25.307617 (+ 1117us) server_negotiation.cc:407] Connection header received
0812 17:52:25.309872 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:25.309876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:25.309941 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:25.310023 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:25.311286 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.312292 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.313645 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.313901 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.314001 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.315367 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:25.315371 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:25.315373 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:25.315388 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:25.315407 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:25.315415 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:25.315479 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:25.315734 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 17:52:25.315799 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260812 17:52:25.317153 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.306594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:37263 (local address 127.13.38.3:39871)
0812 17:52:25.307380 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:25.307460 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:25.309515 (+ 2055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:25.310308 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:25.310318 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:25.310417 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:25.311090 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.311110 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.312468 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.312472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.313457 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.313468 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.314118 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.314122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.314333 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.315068 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:52:25.315070 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:25.315118 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:52:25.315197 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:25.315502 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:25.315510 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:25.315515 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 17:52:25.315703 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 17:52:25.316024 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":158,"threads_started":1}
I20260812 17:52:25.318694 9625 heartbeater.cc:347] Connected to a master server at 127.13.38.62:37263
I20260812 17:52:25.319010 9625 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:25.319851 9625 heartbeater.cc:511] Master 127.13.38.62:37263 requested a full tablet report, sending...
I20260812 17:52:25.321504 8921 ts_manager.cc:194] Registered new tserver with Master: f399674d43804bb5a1bfdb0e153e5881 (127.13.38.3:43673)
I20260812 17:52:25.322293 8921 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.38.3:39871
I20260812 17:52:25.323714 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:52:25.330834 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.326779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.13.38.62:37263)
0812 17:52:25.326914 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:25.326923 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:25.326959 (+ 36us) server_negotiation.cc:407] Connection header received
0812 17:52:25.327139 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:25.327142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:25.327197 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:25.327279 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:25.327637 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.328163 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.329047 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.329275 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.329388 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.330366 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:25.330370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:25.330373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:25.330393 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:25.330414 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:25.330422 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:25.330507 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:25.330604 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 17:52:25.330669 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 17:52:25.334229 8921 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:52:25.337678 8921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:52:25.353953 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.347349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:37213 (local address 127.0.0.1:57046)
0812 17:52:25.348213 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:25.348274 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:25.348422 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:25.349502 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:25.349509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:25.349540 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:25.349679 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.349685 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.351228 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.351232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.352402 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.352412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.353268 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.353272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.353461 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.353512 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 17:52:25.353645 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 17:52:25.353778 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":45,"thread_start_us":109,"threads_started":1}
I20260812 17:52:25.354018 9649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.347477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57046 (local address 127.13.38.1:37213)
0812 17:52:25.348960 (+ 1483us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:25.348971 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:25.348999 (+ 28us) server_negotiation.cc:407] Connection header received
0812 17:52:25.349088 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:25.349096 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:25.349315 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:25.349455 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:25.349900 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.351090 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.352546 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.353147 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.353263 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.353479 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:25.353694 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 17:52:25.353812 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1337,"thread_start_us":125,"threads_started":1}
I20260812 17:52:25.354800 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.348208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:43673 (local address 127.0.0.1:35792)
0812 17:52:25.349683 (+ 1475us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:25.349694 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:25.349815 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:25.350439 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:25.350443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:25.350464 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:25.350568 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.350579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.351836 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.351839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.353020 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.353030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.354335 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.354338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.354454 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.354485 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 17:52:25.354576 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 17:52:25.354653 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1415,"thread_start_us":95,"threads_started":1}
I20260812 17:52:25.355404 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.348398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35792 (local address 127.13.38.3:43673)
0812 17:52:25.348822 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:25.348832 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:25.349814 (+ 982us) server_negotiation.cc:407] Connection header received
0812 17:52:25.349980 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:25.349986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:25.350185 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:25.350460 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:25.350703 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.351709 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.353154 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.353946 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.354061 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.354263 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:25.355105 (+ 842us) server_negotiation.cc:299] Negotiation successful
0812 17:52:25.355216 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":163,"threads_started":1}
I20260812 17:52:25.356261 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.347836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:43509 (local address 127.0.0.1:47936)
0812 17:52:25.348928 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:25.348943 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:25.349044 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:25.351449 (+ 2405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:25.351454 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:25.351472 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:25.351591 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.351599 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.353155 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.353158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.354312 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.354324 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.355819 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.355823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.355929 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.355954 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 17:52:25.356038 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 17:52:25.356108 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":36,"thread_start_us":112,"threads_started":1}
I20260812 17:52:25.356334 9656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.348521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47936 (local address 127.13.38.2:43509)
0812 17:52:25.350101 (+ 1580us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:25.350114 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:25.350138 (+ 24us) server_negotiation.cc:407] Connection header received
0812 17:52:25.350216 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:25.350221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:25.350420 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:25.350565 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:25.351753 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.353008 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.354471 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.355702 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.355819 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.356016 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:25.356071 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 17:52:25.356164 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1471,"thread_start_us":129,"threads_started":1}
I20260812 17:52:25.357962 9130 tablet_service.cc:1467] Processing CreateTablet for tablet 3357888a4b964b7e8b96c261de86c1c3 (DEFAULT_TABLE table=test-table [id=4ca493064c5f4633a5a3de18444bce7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:25.358007 9509 tablet_service.cc:1467] Processing CreateTablet for tablet 3357888a4b964b7e8b96c261de86c1c3 (DEFAULT_TABLE table=test-table [id=4ca493064c5f4633a5a3de18444bce7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:25.359133 9130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3357888a4b964b7e8b96c261de86c1c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:25.359131 9509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3357888a4b964b7e8b96c261de86c1c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:25.362754 9323 tablet_service.cc:1467] Processing CreateTablet for tablet 3357888a4b964b7e8b96c261de86c1c3 (DEFAULT_TABLE table=test-table [id=4ca493064c5f4633a5a3de18444bce7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:25.364081 9323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3357888a4b964b7e8b96c261de86c1c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:25.366539 9657 tablet_bootstrap.cc:492] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881: Bootstrap starting.
I20260812 17:52:25.368326 9658 tablet_bootstrap.cc:492] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c: Bootstrap starting.
I20260812 17:52:25.369079 9657 tablet_bootstrap.cc:654] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:25.369976 9657 log.cc:824] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:25.370683 9658 tablet_bootstrap.cc:654] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:25.371748 9658 log.cc:824] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:25.372352 9663 tablet_bootstrap.cc:492] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4: Bootstrap starting.
I20260812 17:52:25.372622 9657 tablet_bootstrap.cc:492] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881: No bootstrap required, opened a new log
I20260812 17:52:25.372852 9657 ts_tablet_manager.cc:1397] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 17:52:25.374146 9658 tablet_bootstrap.cc:492] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c: No bootstrap required, opened a new log
I20260812 17:52:25.374387 9658 ts_tablet_manager.cc:1397] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:52:25.374985 9663 tablet_bootstrap.cc:654] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:25.376003 9663 log.cc:824] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:25.377637 9657 raft_consensus.cc:348] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37213 } } peers { permanent_uuid: "f399674d43804bb5a1bfdb0e153e5881" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 43673 } } peers { permanent_uuid: "9ac7f566bbaa4ba98329796135a9cab4" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43509 } }
I20260812 17:52:25.377947 9657 raft_consensus.cc:374] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:25.378022 9657 raft_consensus.cc:729] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f399674d43804bb5a1bfdb0e153e5881, State: Initialized, Role: FOLLOWER
I20260812 17:52:25.378505 9657 consensus_queue.cc:260] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37213 } } peers { permanent_uuid: "f399674d43804bb5a1bfdb0e153e5881" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 43673 } } peers { permanent_uuid: "9ac7f566bbaa4ba98329796135a9cab4" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43509 } }
I20260812 17:52:25.378777 9663 tablet_bootstrap.cc:492] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4: No bootstrap required, opened a new log
I20260812 17:52:25.378801 9658 raft_consensus.cc:348] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37213 } } peers { permanent_uuid: "f399674d43804bb5a1bfdb0e153e5881" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 43673 } } peers { permanent_uuid: "9ac7f566bbaa4ba98329796135a9cab4" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43509 } }
I20260812 17:52:25.379012 9663 ts_tablet_manager.cc:1397] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 17:52:25.379086 9658 raft_consensus.cc:374] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:25.379156 9658 raft_consensus.cc:729] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94f6ec8a30ca41d59f49cf2b8075ec6c, State: Initialized, Role: FOLLOWER
I20260812 17:52:25.379689 9658 consensus_queue.cc:260] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37213 } } peers { permanent_uuid: "f399674d43804bb5a1bfdb0e153e5881" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 43673 } } peers { permanent_uuid: "9ac7f566bbaa4ba98329796135a9cab4" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43509 } }
I20260812 17:52:25.379958 9657 ts_tablet_manager.cc:1428] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 17:52:25.380357 9625 heartbeater.cc:503] Master 127.13.38.62:37263 was elected leader, sending a full tablet report...
I20260812 17:52:25.380421 9231 heartbeater.cc:503] Master 127.13.38.62:37263 was elected leader, sending a full tablet report...
I20260812 17:52:25.380752 9658 ts_tablet_manager.cc:1428] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 17:52:25.384272 9663 raft_consensus.cc:348] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37213 } } peers { permanent_uuid: "f399674d43804bb5a1bfdb0e153e5881" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 43673 } } peers { permanent_uuid: "9ac7f566bbaa4ba98329796135a9cab4" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43509 } }
I20260812 17:52:25.384676 9663 raft_consensus.cc:374] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:25.384768 9663 raft_consensus.cc:729] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac7f566bbaa4ba98329796135a9cab4, State: Initialized, Role: FOLLOWER
I20260812 17:52:25.385295 9663 consensus_queue.cc:260] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37213 } } peers { permanent_uuid: "f399674d43804bb5a1bfdb0e153e5881" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 43673 } } peers { permanent_uuid: "9ac7f566bbaa4ba98329796135a9cab4" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43509 } }
I20260812 17:52:25.386264 9423 heartbeater.cc:503] Master 127.13.38.62:37263 was elected leader, sending a full tablet report...
I20260812 17:52:25.387282 9663 ts_tablet_manager.cc:1428] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260812 17:52:25.428829 9668 raft_consensus.cc:482] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:52:25.429037 9668 raft_consensus.cc:504] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37213 } } peers { permanent_uuid: "f399674d43804bb5a1bfdb0e153e5881" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 43673 } } peers { permanent_uuid: "9ac7f566bbaa4ba98329796135a9cab4" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43509 } }
I20260812 17:52:25.430235 9668 leader_election.cc:288] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94f6ec8a30ca41d59f49cf2b8075ec6c (127.13.38.1:37213), 9ac7f566bbaa4ba98329796135a9cab4 (127.13.38.2:43509)
I20260812 17:52:25.434480 9649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.430785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:50695 (local address 127.13.38.1:37213)
0812 17:52:25.430945 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:25.430950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:25.430966 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:52:25.431026 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:25.431029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:25.431097 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:25.431180 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:25.431608 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.432216 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.433467 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.433866 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.433963 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.434059 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:25.434251 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 17:52:25.434335 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 17:52:25.434513 9656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.430767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:33377 (local address 127.13.38.2:43509)
0812 17:52:25.431016 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:25.431020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:25.431217 (+ 197us) server_negotiation.cc:407] Connection header received
0812 17:52:25.431280 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:25.431282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:25.431331 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:25.431422 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:25.431766 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.432286 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.433481 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.433890 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.434023 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.434086 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:25.434298 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 17:52:25.434377 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260812 17:52:25.434513 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.430528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:37213 (local address 127.13.38.3:50695)
0812 17:52:25.430661 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:25.430680 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:25.430781 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:25.431242 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:25.431246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:25.431271 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:25.431420 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.431428 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.432335 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.432341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.433329 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.433341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.433996 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.434000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.434114 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.434137 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 17:52:25.434208 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 17:52:25.434304 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":78}
I20260812 17:52:25.434743 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.430528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:43509 (local address 127.13.38.3:33377)
0812 17:52:25.431037 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:25.431051 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:25.431171 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:25.431453 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:25.431456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:25.431473 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:25.431645 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.431653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.432410 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.432413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.433330 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:25.433340 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.434059 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.434064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:25.434151 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.434168 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 17:52:25.434264 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 17:52:25.434333 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":151,"thread_start_us":142,"threads_started":1}
I20260812 17:52:25.435269 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3357888a4b964b7e8b96c261de86c1c3" candidate_uuid: "f399674d43804bb5a1bfdb0e153e5881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c" is_pre_election: true
I20260812 17:52:25.435429 9356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3357888a4b964b7e8b96c261de86c1c3" candidate_uuid: "f399674d43804bb5a1bfdb0e153e5881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac7f566bbaa4ba98329796135a9cab4" is_pre_election: true
I20260812 17:52:25.435635 9161 raft_consensus.cc:2454] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f399674d43804bb5a1bfdb0e153e5881 in term 0.
I20260812 17:52:25.435761 9356 raft_consensus.cc:2454] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f399674d43804bb5a1bfdb0e153e5881 in term 0.
I20260812 17:52:25.436265 9453 leader_election.cc:302] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94f6ec8a30ca41d59f49cf2b8075ec6c, f399674d43804bb5a1bfdb0e153e5881; no voters:
I20260812 17:52:25.436580 9668 raft_consensus.cc:2781] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:52:25.436666 9668 raft_consensus.cc:482] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:52:25.436698 9668 raft_consensus.cc:3037] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:25.437776 9668 raft_consensus.cc:504] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37213 } } peers { permanent_uuid: "f399674d43804bb5a1bfdb0e153e5881" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 43673 } } peers { permanent_uuid: "9ac7f566bbaa4ba98329796135a9cab4" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43509 } }
I20260812 17:52:25.438180 9668 leader_election.cc:288] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [CANDIDATE]: Term 1 election: Requested vote from peers 94f6ec8a30ca41d59f49cf2b8075ec6c (127.13.38.1:37213), 9ac7f566bbaa4ba98329796135a9cab4 (127.13.38.2:43509)
I20260812 17:52:25.438581 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3357888a4b964b7e8b96c261de86c1c3" candidate_uuid: "f399674d43804bb5a1bfdb0e153e5881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c"
I20260812 17:52:25.438591 9356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3357888a4b964b7e8b96c261de86c1c3" candidate_uuid: "f399674d43804bb5a1bfdb0e153e5881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac7f566bbaa4ba98329796135a9cab4"
I20260812 17:52:25.438748 9161 raft_consensus.cc:3037] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:25.438755 9356 raft_consensus.cc:3037] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:25.439977 9356 raft_consensus.cc:2454] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f399674d43804bb5a1bfdb0e153e5881 in term 1.
I20260812 17:52:25.439994 9161 raft_consensus.cc:2454] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f399674d43804bb5a1bfdb0e153e5881 in term 1.
I20260812 17:52:25.440346 9449 leader_election.cc:302] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ac7f566bbaa4ba98329796135a9cab4, f399674d43804bb5a1bfdb0e153e5881; no voters:
I20260812 17:52:25.440539 9668 raft_consensus.cc:2781] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:25.440802 9668 raft_consensus.cc:686] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 1 LEADER]: Becoming Leader. State: Replica: f399674d43804bb5a1bfdb0e153e5881, State: Running, Role: LEADER
I20260812 17:52:25.441224 9668 consensus_queue.cc:237] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37213 } } peers { permanent_uuid: "f399674d43804bb5a1bfdb0e153e5881" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 43673 } } peers { permanent_uuid: "9ac7f566bbaa4ba98329796135a9cab4" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43509 } }
I20260812 17:52:25.444628 8921 catalog_manager.cc:5495] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 reported cstate change: term changed from 0 to 1, leader changed from <none> to f399674d43804bb5a1bfdb0e153e5881 (127.13.38.3). New cstate: current_term: 1 leader_uuid: "f399674d43804bb5a1bfdb0e153e5881" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f399674d43804bb5a1bfdb0e153e5881" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 43673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ac7f566bbaa4ba98329796135a9cab4" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43509 } health_report { overall_health: UNKNOWN } } }
I20260812 17:52:25.480195 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.476219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35806 (local address 127.13.38.3:43673)
0812 17:52:25.476361 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:25.476366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:25.476381 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:52:25.476563 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:25.476567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:25.476635 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:25.476758 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:52:25.477061 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.477576 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.478449 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:25.478680 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:25.478795 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:25.479153 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:52:25.479711 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:52:25.479847 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:25.479939 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 17:52:25.480004 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 17:52:25.487624 9356 raft_consensus.cc:1261] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4 [term 1 FOLLOWER]: Refusing update from remote peer f399674d43804bb5a1bfdb0e153e5881: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:52:25.487624 9161 raft_consensus.cc:1261] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c [term 1 FOLLOWER]: Refusing update from remote peer f399674d43804bb5a1bfdb0e153e5881: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:52:25.488513 9674 consensus_queue.cc:1035] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7f566bbaa4ba98329796135a9cab4" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:52:25.488794 9668 consensus_queue.cc:1035] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94f6ec8a30ca41d59f49cf2b8075ec6c" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:52:25.510283 8921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:52:25.510502 8921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:52:25.513533 8921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f94319a79d7f4a8ab339e5959d082e6f: Sending DeleteTablet for 3 replicas of tablet 3357888a4b964b7e8b96c261de86c1c3
I20260812 17:52:25.514724 9130 tablet_service.cc:1514] Processing DeleteTablet for tablet 3357888a4b964b7e8b96c261de86c1c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:25 UTC) from {username='slave'} at 127.0.0.1:57046
I20260812 17:52:25.514833 9509 tablet_service.cc:1514] Processing DeleteTablet for tablet 3357888a4b964b7e8b96c261de86c1c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:25 UTC) from {username='slave'} at 127.0.0.1:35792
I20260812 17:52:25.515468 9692 tablet_replica.cc:330] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c: stopping tablet replica
I20260812 17:52:25.515475 9323 tablet_service.cc:1514] Processing DeleteTablet for tablet 3357888a4b964b7e8b96c261de86c1c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:25 UTC) from {username='slave'} at 127.0.0.1:47936
I20260812 17:52:25.515808 9693 tablet_replica.cc:330] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881: stopping tablet replica
I20260812 17:52:25.516115 9694 tablet_replica.cc:330] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4: stopping tablet replica
I20260812 17:52:25.516218 9693 raft_consensus.cc:2229] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:52:25.516448 9694 raft_consensus.cc:2229] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:52:25.516559 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 9032
I20260812 17:52:25.516706 9692 raft_consensus.cc:2229] T 3357888a4b964b7e8b96c261de86c1c3 P 94f6ec8a30ca41d59f49cf2b8075ec6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:52:25.516741 9693 raft_consensus.cc:2258] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:52:25.518998 9693 ts_tablet_manager.cc:1905] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:52:25.523089 9693 ts_tablet_manager.cc:1918] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:52:25.523324 9693 log.cc:1196] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/3357888a4b964b7e8b96c261de86c1c3
I20260812 17:52:25.523789 9693 ts_tablet_manager.cc:1939] T 3357888a4b964b7e8b96c261de86c1c3 P f399674d43804bb5a1bfdb0e153e5881: Deleting consensus metadata
I20260812 17:52:25.525547 8910 catalog_manager.cc:4841] TS f399674d43804bb5a1bfdb0e153e5881 (127.13.38.3:43673): tablet 3357888a4b964b7e8b96c261de86c1c3 (table test-table [id=4ca493064c5f4633a5a3de18444bce7e]) successfully deleted
I20260812 17:52:25.543432 9694 raft_consensus.cc:2258] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:52:25.545383 9694 ts_tablet_manager.cc:1905] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:52:25.548796 9694 ts_tablet_manager.cc:1918] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:52:25.548923 9694 log.cc:1196] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/3357888a4b964b7e8b96c261de86c1c3
I20260812 17:52:25.549237 9694 ts_tablet_manager.cc:1939] T 3357888a4b964b7e8b96c261de86c1c3 P 9ac7f566bbaa4ba98329796135a9cab4: Deleting consensus metadata
I20260812 17:52:25.550773 8905 catalog_manager.cc:4841] TS 9ac7f566bbaa4ba98329796135a9cab4 (127.13.38.2:43509): tablet 3357888a4b964b7e8b96c261de86c1c3 (table test-table [id=4ca493064c5f4633a5a3de18444bce7e]) successfully deleted
W20260812 17:52:25.551466 8910 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:37213 (error 108)
I20260812 17:52:25.552668 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:25.552350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:37213 (local address 127.0.0.1:57054)
0812 17:52:25.552509 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:25.552584 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:37213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 17:52:25.552924 8910 catalog_manager.cc:4584] TS 94f6ec8a30ca41d59f49cf2b8075ec6c (127.13.38.1:37213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3357888a4b964b7e8b96c261de86c1c3: Network error: Client connection negotiation failed: client connection to 127.13.38.1:37213: connect: Connection refused (error 111)
I20260812 17:52:25.553542 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 9235
I20260812 17:52:25.570955 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 9427
I20260812 17:52:25.587025 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 8883
2026-08-12T17:52:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1297 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:25 dist-test-slave-sb2z krb5kdc[9702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:25 dist-test-slave-sb2z krb5kdc[9702](info): set up 2 sockets
Aug 12 17:52:25 dist-test-slave-sb2z krb5kdc[9702](info): commencing operation
krb5kdc: starting...
W20260812 17:52:26.914816 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:27 dist-test-slave-sb2z krb5kdc[9702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:52:27.072153 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:33063
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:33063
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:27.196588 9716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:27.196981 9716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:27.197064 9716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:27.201401 9716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:27.201525 9716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:27.201565 9716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:27.201606 9716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:27.201643 9716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:27.207515 9716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:33063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:33063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:27.208778 9716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:27.209928 9716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:27.219599 9724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:27.219617 9723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:27.219623 9726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:27.220304 9716 server_base.cc:1034] running on GCE node
I20260812 17:52:27.221168 9716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:27.222504 9716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:27.223738 9716 hybrid_clock.cc:648] HybridClock initialized: now 1786557147223719 us; error 35 us; skew 500 ppm
Aug 12 17:52:27 dist-test-slave-sb2z krb5kdc[9702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557147, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:27.230686 9716 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:52:27.232085 9716 webserver.cc:460] Webserver started at http://127.13.38.62:33109/ using document root <none> and password file <none>
I20260812 17:52:27.232764 9716 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:27.232882 9716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:27.233156 9716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:27.235507 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "91dfcafa1f1b4e0280d9a418ad0dc34e"
format_stamp: "Formatted at 2026-08-12 17:52:27 on dist-test-slave-sb2z"
server_key: "83edc4a7e4a4ad29c94dad91bf902f14"
server_key_iv: "4ab0fedcc3c495a7a29de5f5ca3119f0"
server_key_version: "encryptionkey@0"
I20260812 17:52:27.236276 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "91dfcafa1f1b4e0280d9a418ad0dc34e"
format_stamp: "Formatted at 2026-08-12 17:52:27 on dist-test-slave-sb2z"
server_key: "83edc4a7e4a4ad29c94dad91bf902f14"
server_key_iv: "4ab0fedcc3c495a7a29de5f5ca3119f0"
server_key_version: "encryptionkey@0"
I20260812 17:52:27.241168 9716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 17:52:27.244477 9741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:27.246130 9716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:52:27.246335 9716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "91dfcafa1f1b4e0280d9a418ad0dc34e"
format_stamp: "Formatted at 2026-08-12 17:52:27 on dist-test-slave-sb2z"
server_key: "83edc4a7e4a4ad29c94dad91bf902f14"
server_key_iv: "4ab0fedcc3c495a7a29de5f5ca3119f0"
server_key_version: "encryptionkey@0"
I20260812 17:52:27.246498 9716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:27.269137 9716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:27.273023 9716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:27.273257 9716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:27.281150 9716 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:33063
I20260812 17:52:27.281180 9836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:33063 every 8 connection(s)
I20260812 17:52:27.282259 9716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:27.285400 9841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:27.291510 9841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e: Bootstrap starting.
I20260812 17:52:27.291774 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 9716
I20260812 17:52:27.291922 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:27.292235 13464 external_mini_cluster.cc:1428] Setting key a9c7ee8dce8e8703e36787bb95ba053e
I20260812 17:52:27.294523 9841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:27.295338 9841 log.cc:824] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:27.297394 9841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e: No bootstrap required, opened a new log
I20260812 17:52:27.301487 9841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91dfcafa1f1b4e0280d9a418ad0dc34e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33063 } }
Aug 12 17:52:27 dist-test-slave-sb2z krb5kdc[9702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:52:27.301860 9841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:27.301927 9841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91dfcafa1f1b4e0280d9a418ad0dc34e, State: Initialized, Role: FOLLOWER
I20260812 17:52:27.302424 9841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91dfcafa1f1b4e0280d9a418ad0dc34e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33063 } }
I20260812 17:52:27.302567 9841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:27.302656 9841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:27.302785 9841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:27.304142 9841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91dfcafa1f1b4e0280d9a418ad0dc34e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33063 } }
I20260812 17:52:27.304562 9841 leader_election.cc:302] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91dfcafa1f1b4e0280d9a418ad0dc34e; no voters:
I20260812 17:52:27.304965 9841 leader_election.cc:288] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:27.305068 9847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:27.305253 9847 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [term 1 LEADER]: Becoming Leader. State: Replica: 91dfcafa1f1b4e0280d9a418ad0dc34e, State: Running, Role: LEADER
I20260812 17:52:27.305572 9847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91dfcafa1f1b4e0280d9a418ad0dc34e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33063 } }
I20260812 17:52:27.306427 9841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:27.306860 9847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91dfcafa1f1b4e0280d9a418ad0dc34e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91dfcafa1f1b4e0280d9a418ad0dc34e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33063 } } }
I20260812 17:52:27.306981 9847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:27.307199 9847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91dfcafa1f1b4e0280d9a418ad0dc34e. Latest consensus state: current_term: 1 leader_uuid: "91dfcafa1f1b4e0280d9a418ad0dc34e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91dfcafa1f1b4e0280d9a418ad0dc34e" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33063 } } }
I20260812 17:52:27.307305 9847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:27.307963 9852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:27.309090 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:27.293777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37492 (local address 127.13.38.62:33063)
0812 17:52:27.294296 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:27.294312 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:27.294351 (+ 39us) server_negotiation.cc:407] Connection header received
0812 17:52:27.295104 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:27.295124 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:27.295564 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:27.295937 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:27.296804 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:27.297853 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:27.298730 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:27.299219 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:27.299414 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:27.302832 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:27.302839 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:27.302876 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:27.302922 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:27.305145 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:27.305684 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:27.305694 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:27.305703 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:27.305846 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:27.306159 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:27.306162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:27.306164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:27.306656 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:27.306864 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:27.307727 (+ 863us) server_negotiation.cc:299] Negotiation successful
0812 17:52:27.308069 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":151,"threads_started":1}
I20260812 17:52:27.312475 9852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:27.319588 9852 catalog_manager.cc:1347] Generated new cluster ID: 406330714f934e2baa71adcffba239e4
I20260812 17:52:27.319703 9852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:27.329723 9852 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:27.331111 9852 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:27.339162 9852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e: Generated new TSK 0
I20260812 17:52:27.340097 9852 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:52:27.383162 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:33063
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:27.514768 9873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:27.515110 9873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:27.515202 9873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:27.519662 9873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:27.519747 9873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:27.519820 9873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:52:27.526221 9873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:33063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:27.527415 9873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:27.528607 9873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:27.539575 9880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:27.539726 9873 server_base.cc:1034] running on GCE node
W20260812 17:52:27.539495 9879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:27.539579 9885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:27.540367 9873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:27.541158 9873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:27.542372 9873 hybrid_clock.cc:648] HybridClock initialized: now 1786557147542354 us; error 33 us; skew 500 ppm
Aug 12 17:52:27 dist-test-slave-sb2z krb5kdc[9702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557147, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:27.545583 9873 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:52:27.546649 9873 webserver.cc:460] Webserver started at http://127.13.38.1:46135/ using document root <none> and password file <none>
I20260812 17:52:27.547326 9873 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:27.547428 9873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:27.547729 9873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:27.549997 9873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "5ab85ece6d3d4a6889265e4e08063522"
format_stamp: "Formatted at 2026-08-12 17:52:27 on dist-test-slave-sb2z"
server_key: "7a825a9e20ec24784089e48d6da61a5d"
server_key_iv: "c8ef0097ef201171dca49c5f56bab48f"
server_key_version: "encryptionkey@0"
I20260812 17:52:27.550736 9873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "5ab85ece6d3d4a6889265e4e08063522"
format_stamp: "Formatted at 2026-08-12 17:52:27 on dist-test-slave-sb2z"
server_key: "7a825a9e20ec24784089e48d6da61a5d"
server_key_iv: "c8ef0097ef201171dca49c5f56bab48f"
server_key_version: "encryptionkey@0"
I20260812 17:52:27.554988 9873 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:52:27.557982 9893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:27.559648 9873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:27.559813 9873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "5ab85ece6d3d4a6889265e4e08063522"
format_stamp: "Formatted at 2026-08-12 17:52:27 on dist-test-slave-sb2z"
server_key: "7a825a9e20ec24784089e48d6da61a5d"
server_key_iv: "c8ef0097ef201171dca49c5f56bab48f"
server_key_version: "encryptionkey@0"
I20260812 17:52:27.559966 9873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:27.575116 9873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:27.579134 9873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:27.579381 9873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:27.580117 9873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:27.581063 9873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:27.581139 9873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:27.581218 9873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:27.581260 9873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:27.591940 9873 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:39053
I20260812 17:52:27.592051 10077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:39053 every 8 connection(s)
I20260812 17:52:27.592972 9873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:52:27.602406 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 9873
I20260812 17:52:27.602597 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:52:27.602931 13464 external_mini_cluster.cc:1428] Setting key 50a870b40ac60e526aa3cea7478c3077
Aug 12 17:52:27 dist-test-slave-sb2z krb5kdc[9702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557147, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:52:27.609675 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:27.595199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:43375 (local address 127.13.38.62:33063)
0812 17:52:27.595351 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:27.595355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:27.596735 (+ 1380us) server_negotiation.cc:407] Connection header received
0812 17:52:27.597716 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:27.597719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:27.597775 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:27.597866 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:27.599182 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:27.599840 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:27.601133 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:27.601369 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:27.601463 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:27.604706 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:27.604709 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:27.604711 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:27.604756 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:27.606810 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:27.607672 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:27.607677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:27.607679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:27.607789 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:27.608235 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:27.608239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:27.608244 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:27.608548 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:27.608669 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:27.609319 (+ 650us) server_negotiation.cc:299] Negotiation successful
0812 17:52:27.609480 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 17:52:27.610641 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:27.595730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:33063 (local address 127.13.38.1:43375)
0812 17:52:27.596557 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:27.596607 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:27.597455 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:27.598095 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:27.598103 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:27.598505 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:27.599010 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:27.599025 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:27.599978 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:27.599981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:27.600963 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:27.600976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:27.601550 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:27.601553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:27.601713 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:27.602471 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:52:27.602500 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:27.604509 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:27.606978 (+ 2469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:27.606985 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:27.607002 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:27.607465 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:27.607929 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:27.607933 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:27.607936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:27.608112 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:27.608669 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:27.608679 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:27.609040 (+ 361us) client_negotiation.cc:769] Sending connection context
0812 17:52:27.609257 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 17:52:27.609651 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":138,"threads_started":1}
I20260812 17:52:27.612099 10078 heartbeater.cc:347] Connected to a master server at 127.13.38.62:33063
I20260812 17:52:27.612360 10078 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:27.612941 10078 heartbeater.cc:511] Master 127.13.38.62:33063 requested a full tablet report, sending...
I20260812 17:52:27.614677 9784 ts_manager.cc:194] Registered new tserver with Master: 5ab85ece6d3d4a6889265e4e08063522 (127.13.38.1:39053)
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
I20260812 17:52:27.616274 9784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:43375
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:52:27.640062 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:33063
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:27.766701 10086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:27.767005 10086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:27.767052 10086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:27.771515 10086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:27.771611 10086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:27.771706 10086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:52:27.777493 10086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:33063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:27.778710 10086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:27.779855 10086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:27.791020 10098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:27.791028 10096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:27.791268 10086 server_base.cc:1034] running on GCE node
W20260812 17:52:27.791054 10100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:27.791960 10086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:27.792738 10086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:27.793921 10086 hybrid_clock.cc:648] HybridClock initialized: now 1786557147793903 us; error 36 us; skew 500 ppm
Aug 12 17:52:27 dist-test-slave-sb2z krb5kdc[9702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557147, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:27.797266 10086 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:52:27.798321 10086 webserver.cc:460] Webserver started at http://127.13.38.2:35741/ using document root <none> and password file <none>
I20260812 17:52:27.798990 10086 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:27.799096 10086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:27.799355 10086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:27.801693 10086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "fd55010d64ae417a8fa2a92621c19004"
format_stamp: "Formatted at 2026-08-12 17:52:27 on dist-test-slave-sb2z"
server_key: "9b08bd95a85c4f2a5f229f8b011ca8d9"
server_key_iv: "dc086d71cac39b88e0e2611e64d0edde"
server_key_version: "encryptionkey@0"
I20260812 17:52:27.802320 10086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "fd55010d64ae417a8fa2a92621c19004"
format_stamp: "Formatted at 2026-08-12 17:52:27 on dist-test-slave-sb2z"
server_key: "9b08bd95a85c4f2a5f229f8b011ca8d9"
server_key_iv: "dc086d71cac39b88e0e2611e64d0edde"
server_key_version: "encryptionkey@0"
I20260812 17:52:27.806447 10086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:52:27.809866 10110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:27.811755 10086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:27.811939 10086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "fd55010d64ae417a8fa2a92621c19004"
format_stamp: "Formatted at 2026-08-12 17:52:27 on dist-test-slave-sb2z"
server_key: "9b08bd95a85c4f2a5f229f8b011ca8d9"
server_key_iv: "dc086d71cac39b88e0e2611e64d0edde"
server_key_version: "encryptionkey@0"
I20260812 17:52:27.812121 10086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:27.829097 10086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:27.832875 10086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:27.833173 10086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:27.833860 10086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:27.834782 10086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:27.834860 10086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:27.834941 10086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:27.834986 10086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:27.846591 10086 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:33321
I20260812 17:52:27.846616 10288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:33321 every 8 connection(s)
I20260812 17:52:27.847656 10086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:52:27.849413 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 10086
I20260812 17:52:27.849526 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:52:27.849825 13464 external_mini_cluster.cc:1428] Setting key b12297bf827665007508b5a12b3682f3
Aug 12 17:52:27 dist-test-slave-sb2z krb5kdc[9702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557147, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:52:27.864784 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:27.850378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:58139 (local address 127.13.38.62:33063)
0812 17:52:27.850527 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:27.850532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:27.851457 (+ 925us) server_negotiation.cc:407] Connection header received
0812 17:52:27.852429 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:27.852436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:27.852513 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:27.852611 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:27.854002 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:27.854532 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:27.855983 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:27.856391 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:27.856553 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:27.859864 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:27.859870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:27.859875 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:27.859919 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:27.861802 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:27.862590 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:27.862594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:27.862595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:27.862691 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:27.863221 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:27.863233 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:27.863234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:27.863454 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:27.863816 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:27.864519 (+ 703us) server_negotiation.cc:299] Negotiation successful
0812 17:52:27.864650 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
I20260812 17:52:27.865893 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:27.850338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:33063 (local address 127.13.38.2:58139)
0812 17:52:27.851284 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:27.851344 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:27.852217 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:27.852745 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:27.852751 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:27.853141 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:27.853825 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:27.853849 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:27.854679 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:27.854684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:27.855848 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:27.855860 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:27.856609 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:27.856612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:27.856799 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:27.857553 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:52:27.857571 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:27.859627 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:27.862011 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:27.862023 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:27.862054 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:27.862418 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:27.862789 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:27.862791 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:27.862792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:27.863002 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:27.863794 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:27.863801 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:27.864160 (+ 359us) client_negotiation.cc:769] Sending connection context
0812 17:52:27.864401 (+ 241us) client_negotiation.cc:240] Negotiation successful
0812 17:52:27.864732 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":182,"threads_started":1}
I20260812 17:52:27.867458 10290 heartbeater.cc:347] Connected to a master server at 127.13.38.62:33063
I20260812 17:52:27.867777 10290 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:27.868350 10290 heartbeater.cc:511] Master 127.13.38.62:33063 requested a full tablet report, sending...
I20260812 17:52:27.869562 9784 ts_manager.cc:194] Registered new tserver with Master: fd55010d64ae417a8fa2a92621c19004 (127.13.38.2:33321)
I20260812 17:52:27.870231 9784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:58139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:52:27.889212 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:33063
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:28.015089 10298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:28.015466 10298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:28.015599 10298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:28.020303 10298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:28.020411 10298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:28.020514 10298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:52:28.026317 10298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:33063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:28.027678 10298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:28.028923 10298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:28.040268 10311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:28.040365 10309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:28.040306 10308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:28.040975 10298 server_base.cc:1034] running on GCE node
I20260812 17:52:28.041603 10298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:28.042757 10298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:28.044034 10298 hybrid_clock.cc:648] HybridClock initialized: now 1786557148044017 us; error 35 us; skew 500 ppm
Aug 12 17:52:28 dist-test-slave-sb2z krb5kdc[9702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557148, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:28.047320 10298 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:52:28.048761 10298 webserver.cc:460] Webserver started at http://127.13.38.3:43603/ using document root <none> and password file <none>
I20260812 17:52:28.049435 10298 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:28.049544 10298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:28.049816 10298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:28.052542 10298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "9c241d5fc52c4f13836b3e157651de35"
format_stamp: "Formatted at 2026-08-12 17:52:28 on dist-test-slave-sb2z"
server_key: "de9f2ac51448414a02aaa0c26121db99"
server_key_iv: "8dc74cb140acf789407b75a5649886d3"
server_key_version: "encryptionkey@0"
I20260812 17:52:28.053217 10298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "9c241d5fc52c4f13836b3e157651de35"
format_stamp: "Formatted at 2026-08-12 17:52:28 on dist-test-slave-sb2z"
server_key: "de9f2ac51448414a02aaa0c26121db99"
server_key_iv: "8dc74cb140acf789407b75a5649886d3"
server_key_version: "encryptionkey@0"
I20260812 17:52:28.058856 10298 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 17:52:28.063048 10319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:28.064805 10298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 17:52:28.064996 10298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "9c241d5fc52c4f13836b3e157651de35"
format_stamp: "Formatted at 2026-08-12 17:52:28 on dist-test-slave-sb2z"
server_key: "de9f2ac51448414a02aaa0c26121db99"
server_key_iv: "8dc74cb140acf789407b75a5649886d3"
server_key_version: "encryptionkey@0"
I20260812 17:52:28.065151 10298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:28.084610 10298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:28.088339 10298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:28.088586 10298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:28.089254 10298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:28.090226 10298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:28.090304 10298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:28.090381 10298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:28.090427 10298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:28.101303 10298 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:37523
I20260812 17:52:28.101341 10480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:37523 every 8 connection(s)
I20260812 17:52:28.102315 10298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:52:28.109313 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 10298
I20260812 17:52:28.109516 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:52:28.109923 13464 external_mini_cluster.cc:1428] Setting key f4b500ef3e626b6028808ae84b0bf1b3
Aug 12 17:52:28 dist-test-slave-sb2z krb5kdc[9702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557148, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:52:28.118840 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.104540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:38961 (local address 127.13.38.62:33063)
0812 17:52:28.104701 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:28.104705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:28.105769 (+ 1064us) server_negotiation.cc:407] Connection header received
0812 17:52:28.106991 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:28.106994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:28.107076 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:28.107205 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:28.108729 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.109698 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.111092 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.111329 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.111455 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.114462 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:28.114467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:28.114476 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:28.114509 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:28.116371 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:28.117093 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:28.117097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:28.117099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:28.117195 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:28.117586 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:28.117591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:28.117592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:28.117808 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:28.117923 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:28.118511 (+ 588us) server_negotiation.cc:299] Negotiation successful
0812 17:52:28.118662 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 17:52:28.119833 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.104835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:33063 (local address 127.13.38.3:38961)
0812 17:52:28.105597 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:28.105645 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:28.106733 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:28.107414 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:28.107422 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:28.108009 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:28.108560 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.108575 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.109894 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.109897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.110941 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.110954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.111552 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.111558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.111767 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.112500 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:52:28.112525 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:28.114240 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:28.116615 (+ 2375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:28.116623 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:28.116645 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:28.116971 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:28.117299 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:28.117303 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:28.117305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:28.117481 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:28.117928 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:28.117937 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:28.118240 (+ 303us) client_negotiation.cc:769] Sending connection context
0812 17:52:28.118474 (+ 234us) client_negotiation.cc:240] Negotiation successful
0812 17:52:28.118864 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":143,"threads_started":1}
I20260812 17:52:28.121269 10482 heartbeater.cc:347] Connected to a master server at 127.13.38.62:33063
I20260812 17:52:28.121598 10482 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:28.122219 10482 heartbeater.cc:511] Master 127.13.38.62:33063 requested a full tablet report, sending...
I20260812 17:52:28.123811 9782 ts_manager.cc:194] Registered new tserver with Master: 9c241d5fc52c4f13836b3e157651de35 (127.13.38.3:37523)
I20260812 17:52:28.124663 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:52:28.124670 9782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:38961
I20260812 17:52:28.135515 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.126927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.13.38.62:33063)
0812 17:52:28.127081 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:28.127086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:28.127220 (+ 134us) server_negotiation.cc:407] Connection header received
0812 17:52:28.127307 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:28.127310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:28.127347 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:28.127413 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:28.128256 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.128784 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.129685 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.129905 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.130009 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.131762 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:28.131765 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:28.131767 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:28.131793 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:28.133228 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:28.133882 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:28.133885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:28.133887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:28.133983 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:28.134394 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:28.134397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:28.134398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:28.134667 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:28.134801 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:28.135130 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 17:52:28.135300 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 17:52:28.139878 9782 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:52:28.143029 9782 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:52:28.158965 10499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.152787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44100 (local address 127.13.38.2:33321)
0812 17:52:28.153645 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:28.153661 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:28.153689 (+ 28us) server_negotiation.cc:407] Connection header received
0812 17:52:28.153781 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:28.153786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:28.154020 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:28.154212 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:28.154556 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.155512 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.157269 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.158184 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.158298 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.158487 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:28.158692 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 17:52:28.158802 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":118,"threads_started":1}
I20260812 17:52:28.158948 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.152604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:33321 (local address 127.0.0.1:44100)
0812 17:52:28.153069 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:28.153130 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:28.153285 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:28.154205 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:28.154211 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:28.154236 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:28.154404 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.154412 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.155643 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.155649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.157144 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.157154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.158312 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.158315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.158462 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.158507 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 17:52:28.158638 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 17:52:28.158769 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":47,"thread_start_us":100,"threads_started":1}
I20260812 17:52:28.159756 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.153507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:39053 (local address 127.0.0.1:47192)
0812 17:52:28.154120 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:28.154154 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:28.154260 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:28.155491 (+ 1231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:28.155494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:28.155583 (+ 89us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:28.155714 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.155722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.157142 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.157146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.157995 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.158005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.159296 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.159299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.159409 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.159434 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 17:52:28.159509 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 17:52:28.159610 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":3,"thread_start_us":94,"threads_started":1}
I20260812 17:52:28.160269 10501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.153679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47192 (local address 127.13.38.1:39053)
0812 17:52:28.154885 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:28.154892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:28.154911 (+ 19us) server_negotiation.cc:407] Connection header received
0812 17:52:28.154975 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:28.154980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:28.155156 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:28.155319 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:28.155853 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.157009 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.158143 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.158946 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.159072 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.159227 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:28.160003 (+ 776us) server_negotiation.cc:299] Negotiation successful
0812 17:52:28.160122 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1100,"thread_start_us":152,"threads_started":1}
I20260812 17:52:28.160470 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.153869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:37523 (local address 127.0.0.1:51944)
0812 17:52:28.154517 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:28.154539 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:28.154645 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:28.155328 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:28.155331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:28.155352 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:28.155449 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.155455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.157313 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.157315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.158419 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.158437 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.160024 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.160028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.160113 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.160131 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 17:52:28.160227 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 17:52:28.160305 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":6,"thread_start_us":156,"threads_started":1}
I20260812 17:52:28.161556 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.154426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51944 (local address 127.13.38.3:37523)
0812 17:52:28.154823 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:28.154836 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:28.154858 (+ 22us) server_negotiation.cc:407] Connection header received
0812 17:52:28.154940 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:28.154945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:28.155157 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:28.155319 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:28.155601 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.157194 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.158574 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.159646 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.159763 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.159953 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:28.160266 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 17:52:28.161397 (+ 1131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":168,"threads_started":1}
I20260812 17:52:28.163333 9972 tablet_service.cc:1467] Processing CreateTablet for tablet 97ddfabfb8b6471a8d086d268cd60119 (DEFAULT_TABLE table=test-table [id=48166b42962746e986a98b889fdcdce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:28.163331 10186 tablet_service.cc:1467] Processing CreateTablet for tablet 97ddfabfb8b6471a8d086d268cd60119 (DEFAULT_TABLE table=test-table [id=48166b42962746e986a98b889fdcdce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:28.164542 9972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ddfabfb8b6471a8d086d268cd60119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:28.164546 10186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ddfabfb8b6471a8d086d268cd60119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:28.164707 10391 tablet_service.cc:1467] Processing CreateTablet for tablet 97ddfabfb8b6471a8d086d268cd60119 (DEFAULT_TABLE table=test-table [id=48166b42962746e986a98b889fdcdce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:28.165886 10391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ddfabfb8b6471a8d086d268cd60119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:28.171888 10506 tablet_bootstrap.cc:492] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522: Bootstrap starting.
I20260812 17:52:28.174147 10509 tablet_bootstrap.cc:492] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35: Bootstrap starting.
I20260812 17:52:28.174810 10508 tablet_bootstrap.cc:492] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004: Bootstrap starting.
I20260812 17:52:28.175189 10506 tablet_bootstrap.cc:654] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:28.176092 10506 log.cc:824] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:28.176605 10509 tablet_bootstrap.cc:654] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:28.177580 10508 tablet_bootstrap.cc:654] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:28.178105 10509 log.cc:824] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:28.178310 10506 tablet_bootstrap.cc:492] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522: No bootstrap required, opened a new log
I20260812 17:52:28.178480 10508 log.cc:824] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:28.178573 10506 ts_tablet_manager.cc:1397] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 17:52:28.180833 10509 tablet_bootstrap.cc:492] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35: No bootstrap required, opened a new log
I20260812 17:52:28.180974 10508 tablet_bootstrap.cc:492] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004: No bootstrap required, opened a new log
I20260812 17:52:28.181037 10509 ts_tablet_manager.cc:1397] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 17:52:28.181174 10508 ts_tablet_manager.cc:1397] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 17:52:28.183897 10506 raft_consensus.cc:348] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd55010d64ae417a8fa2a92621c19004" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33321 } } peers { permanent_uuid: "5ab85ece6d3d4a6889265e4e08063522" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39053 } } peers { permanent_uuid: "9c241d5fc52c4f13836b3e157651de35" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37523 } }
I20260812 17:52:28.184041 10509 raft_consensus.cc:348] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd55010d64ae417a8fa2a92621c19004" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33321 } } peers { permanent_uuid: "5ab85ece6d3d4a6889265e4e08063522" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39053 } } peers { permanent_uuid: "9c241d5fc52c4f13836b3e157651de35" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37523 } }
I20260812 17:52:28.184243 10506 raft_consensus.cc:374] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:28.184309 10509 raft_consensus.cc:374] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:28.184322 10506 raft_consensus.cc:729] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab85ece6d3d4a6889265e4e08063522, State: Initialized, Role: FOLLOWER
I20260812 17:52:28.184366 10509 raft_consensus.cc:729] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c241d5fc52c4f13836b3e157651de35, State: Initialized, Role: FOLLOWER
I20260812 17:52:28.184837 10509 consensus_queue.cc:260] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd55010d64ae417a8fa2a92621c19004" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33321 } } peers { permanent_uuid: "5ab85ece6d3d4a6889265e4e08063522" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39053 } } peers { permanent_uuid: "9c241d5fc52c4f13836b3e157651de35" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37523 } }
I20260812 17:52:28.184845 10506 consensus_queue.cc:260] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd55010d64ae417a8fa2a92621c19004" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33321 } } peers { permanent_uuid: "5ab85ece6d3d4a6889265e4e08063522" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39053 } } peers { permanent_uuid: "9c241d5fc52c4f13836b3e157651de35" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37523 } }
I20260812 17:52:28.185528 10508 raft_consensus.cc:348] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd55010d64ae417a8fa2a92621c19004" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33321 } } peers { permanent_uuid: "5ab85ece6d3d4a6889265e4e08063522" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39053 } } peers { permanent_uuid: "9c241d5fc52c4f13836b3e157651de35" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37523 } }
I20260812 17:52:28.185828 10078 heartbeater.cc:503] Master 127.13.38.62:33063 was elected leader, sending a full tablet report...
I20260812 17:52:28.185840 10482 heartbeater.cc:503] Master 127.13.38.62:33063 was elected leader, sending a full tablet report...
I20260812 17:52:28.185864 10508 raft_consensus.cc:374] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:28.186019 10509 ts_tablet_manager.cc:1428] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 17:52:28.186034 10508 raft_consensus.cc:729] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd55010d64ae417a8fa2a92621c19004, State: Initialized, Role: FOLLOWER
I20260812 17:52:28.186566 10508 consensus_queue.cc:260] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd55010d64ae417a8fa2a92621c19004" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33321 } } peers { permanent_uuid: "5ab85ece6d3d4a6889265e4e08063522" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39053 } } peers { permanent_uuid: "9c241d5fc52c4f13836b3e157651de35" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37523 } }
I20260812 17:52:28.187551 10506 ts_tablet_manager.cc:1428] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260812 17:52:28.187556 10508 ts_tablet_manager.cc:1428] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 17:52:28.188687 10290 heartbeater.cc:503] Master 127.13.38.62:33063 was elected leader, sending a full tablet report...
I20260812 17:52:28.234301 10514 raft_consensus.cc:482] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:52:28.234580 10514 raft_consensus.cc:504] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd55010d64ae417a8fa2a92621c19004" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33321 } } peers { permanent_uuid: "5ab85ece6d3d4a6889265e4e08063522" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39053 } } peers { permanent_uuid: "9c241d5fc52c4f13836b3e157651de35" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37523 } }
I20260812 17:52:28.236088 10514 leader_election.cc:288] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd55010d64ae417a8fa2a92621c19004 (127.13.38.2:33321), 5ab85ece6d3d4a6889265e4e08063522 (127.13.38.1:39053)
I20260812 17:52:28.241168 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.236434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:33321 (local address 127.13.38.3:54071)
0812 17:52:28.236621 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:28.236635 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:28.236871 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:28.237281 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:28.237284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:28.237307 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:28.237430 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.237437 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.238423 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.238426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.239511 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.239573 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.240502 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.240513 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.240682 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.240716 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 17:52:28.240823 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 17:52:28.240932 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":24}
I20260812 17:52:28.241182 10499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.236628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:54071 (local address 127.13.38.2:33321)
0812 17:52:28.236883 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:28.236888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:28.236906 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:52:28.236984 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:28.236987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:28.237051 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:28.237138 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:28.237591 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.238267 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.239703 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.240341 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.240460 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.240536 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:28.240864 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 17:52:28.240960 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260812 17:52:28.242141 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.236790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:39053 (local address 127.13.38.3:33605)
0812 17:52:28.237113 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:28.237131 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:28.237253 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:28.237643 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:28.237647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:28.237668 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:28.237952 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.237964 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.239079 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.239081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.240552 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.240566 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.241642 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.241646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.241767 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.241828 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 17:52:28.241927 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 17:52:28.242004 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":38,"thread_start_us":141,"threads_started":1}
I20260812 17:52:28.242170 10501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.237252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:33605 (local address 127.13.38.1:39053)
0812 17:52:28.237378 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:28.237383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:28.237398 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:52:28.237452 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:28.237456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:28.237518 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:28.237601 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:28.238121 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.238871 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.240728 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.241477 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.241624 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.241695 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:28.241951 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 17:52:28.242049 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 17:52:28.242192 10227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ddfabfb8b6471a8d086d268cd60119" candidate_uuid: "9c241d5fc52c4f13836b3e157651de35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd55010d64ae417a8fa2a92621c19004" is_pre_election: true
I20260812 17:52:28.242540 10227 raft_consensus.cc:2454] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c241d5fc52c4f13836b3e157651de35 in term 0.
I20260812 17:52:28.243131 10324 leader_election.cc:302] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c241d5fc52c4f13836b3e157651de35, fd55010d64ae417a8fa2a92621c19004; no voters:
I20260812 17:52:28.243155 10006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ddfabfb8b6471a8d086d268cd60119" candidate_uuid: "9c241d5fc52c4f13836b3e157651de35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab85ece6d3d4a6889265e4e08063522" is_pre_election: true
I20260812 17:52:28.243503 10006 raft_consensus.cc:2454] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c241d5fc52c4f13836b3e157651de35 in term 0.
I20260812 17:52:28.243505 10514 raft_consensus.cc:2781] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:52:28.243665 10514 raft_consensus.cc:482] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:52:28.243752 10514 raft_consensus.cc:3037] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:28.245272 10514 raft_consensus.cc:504] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd55010d64ae417a8fa2a92621c19004" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33321 } } peers { permanent_uuid: "5ab85ece6d3d4a6889265e4e08063522" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39053 } } peers { permanent_uuid: "9c241d5fc52c4f13836b3e157651de35" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37523 } }
I20260812 17:52:28.245710 10514 leader_election.cc:288] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [CANDIDATE]: Term 1 election: Requested vote from peers fd55010d64ae417a8fa2a92621c19004 (127.13.38.2:33321), 5ab85ece6d3d4a6889265e4e08063522 (127.13.38.1:39053)
I20260812 17:52:28.246245 10006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ddfabfb8b6471a8d086d268cd60119" candidate_uuid: "9c241d5fc52c4f13836b3e157651de35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab85ece6d3d4a6889265e4e08063522"
I20260812 17:52:28.246261 10227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ddfabfb8b6471a8d086d268cd60119" candidate_uuid: "9c241d5fc52c4f13836b3e157651de35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd55010d64ae417a8fa2a92621c19004"
I20260812 17:52:28.246407 10227 raft_consensus.cc:3037] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:28.246407 10006 raft_consensus.cc:3037] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:28.247951 10006 raft_consensus.cc:2454] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c241d5fc52c4f13836b3e157651de35 in term 1.
I20260812 17:52:28.247951 10227 raft_consensus.cc:2454] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c241d5fc52c4f13836b3e157651de35 in term 1.
I20260812 17:52:28.248354 10324 leader_election.cc:302] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab85ece6d3d4a6889265e4e08063522, 9c241d5fc52c4f13836b3e157651de35; no voters:
I20260812 17:52:28.248615 10514 raft_consensus.cc:2781] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:28.248888 10514 raft_consensus.cc:686] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 1 LEADER]: Becoming Leader. State: Replica: 9c241d5fc52c4f13836b3e157651de35, State: Running, Role: LEADER
I20260812 17:52:28.249325 10514 consensus_queue.cc:237] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd55010d64ae417a8fa2a92621c19004" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33321 } } peers { permanent_uuid: "5ab85ece6d3d4a6889265e4e08063522" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39053 } } peers { permanent_uuid: "9c241d5fc52c4f13836b3e157651de35" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37523 } }
I20260812 17:52:28.253322 9782 catalog_manager.cc:5495] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c241d5fc52c4f13836b3e157651de35 (127.13.38.3). New cstate: current_term: 1 leader_uuid: "9c241d5fc52c4f13836b3e157651de35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd55010d64ae417a8fa2a92621c19004" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ab85ece6d3d4a6889265e4e08063522" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c241d5fc52c4f13836b3e157651de35" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260812 17:52:28.266888 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.262861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51950 (local address 127.13.38.3:37523)
0812 17:52:28.263019 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:28.263024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:28.263042 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:52:28.263283 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:28.263286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:28.263347 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:28.263493 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:52:28.263836 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.264386 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.265326 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.265560 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.265645 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.266008 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:52:28.266434 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:52:28.266575 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:28.266661 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 17:52:28.266699 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 17:52:28.274206 10227 raft_consensus.cc:1261] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004 [term 1 FOLLOWER]: Refusing update from remote peer 9c241d5fc52c4f13836b3e157651de35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:52:28.274204 10006 raft_consensus.cc:1261] T 97ddfabfb8b6471a8d086d268cd60119 P 5ab85ece6d3d4a6889265e4e08063522 [term 1 FOLLOWER]: Refusing update from remote peer 9c241d5fc52c4f13836b3e157651de35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:52:28.275094 10521 consensus_queue.cc:1035] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd55010d64ae417a8fa2a92621c19004" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 33321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:52:28.275341 10514 consensus_queue.cc:1035] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab85ece6d3d4a6889265e4e08063522" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:52:28.284416 10526 mvcc.cc:205] Tried to move back new op lower bound from 7317738079322005504 to 7317738079232319488. Current Snapshot: MvccSnapshot[applied={T|T < 7317738079322005504 or (T in {7317738079322005504})}]
I20260812 17:52:28.293210 9782 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:52:28.293398 9782 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:52:28.295698 10528 mvcc.cc:205] Tried to move back new op lower bound from 7317738079322005504 to 7317738079232319488. Current Snapshot: MvccSnapshot[applied={T|T < 7317738079322005504}]
I20260812 17:52:28.296535 9782 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91dfcafa1f1b4e0280d9a418ad0dc34e: Sending DeleteTablet for 3 replicas of tablet 97ddfabfb8b6471a8d086d268cd60119
I20260812 17:52:28.298000 10186 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ddfabfb8b6471a8d086d268cd60119 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:28 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:44100
I20260812 17:52:28.298576 9972 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ddfabfb8b6471a8d086d268cd60119 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:28 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:47192
I20260812 17:52:28.298624 10538 tablet_replica.cc:330] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004: stopping tablet replica
I20260812 17:52:28.299098 10538 raft_consensus.cc:2229] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:52:28.299091 10391 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ddfabfb8b6471a8d086d268cd60119 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:28 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:51944
I20260812 17:52:28.299640 10538 raft_consensus.cc:2258] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:52:28.300408 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 9873
I20260812 17:52:28.317222 10538 ts_tablet_manager.cc:1905] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:52:28.319851 10541 tablet_replica.cc:330] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35: stopping tablet replica
I20260812 17:52:28.320297 10541 raft_consensus.cc:2229] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:52:28.320756 10541 raft_consensus.cc:2258] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:52:28.323195 10538 ts_tablet_manager.cc:1918] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:52:28.323318 10538 log.cc:1196] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/97ddfabfb8b6471a8d086d268cd60119
I20260812 17:52:28.323709 10538 ts_tablet_manager.cc:1939] T 97ddfabfb8b6471a8d086d268cd60119 P fd55010d64ae417a8fa2a92621c19004: Deleting consensus metadata
I20260812 17:52:28.325271 9744 catalog_manager.cc:4841] TS fd55010d64ae417a8fa2a92621c19004 (127.13.38.2:33321): tablet 97ddfabfb8b6471a8d086d268cd60119 (table test-table [id=48166b42962746e986a98b889fdcdce9]) successfully deleted
I20260812 17:52:28.328209 10541 ts_tablet_manager.cc:1905] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:52:28.332094 10541 ts_tablet_manager.cc:1918] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:52:28.332218 10541 log.cc:1196] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/97ddfabfb8b6471a8d086d268cd60119
I20260812 17:52:28.332584 10541 ts_tablet_manager.cc:1939] T 97ddfabfb8b6471a8d086d268cd60119 P 9c241d5fc52c4f13836b3e157651de35: Deleting consensus metadata
I20260812 17:52:28.334120 9744 catalog_manager.cc:4841] TS 9c241d5fc52c4f13836b3e157651de35 (127.13.38.3:37523): tablet 97ddfabfb8b6471a8d086d268cd60119 (table test-table [id=48166b42962746e986a98b889fdcdce9]) successfully deleted
W20260812 17:52:28.343773 9744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:39053 (error 108)
I20260812 17:52:28.344599 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 10086
I20260812 17:52:28.353755 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.344792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:39053 (local address 127.0.0.1:47202)
0812 17:52:28.353484 (+ 8692us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:28.353620 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:39053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8525}
W20260812 17:52:28.361807 9744 catalog_manager.cc:4584] TS 5ab85ece6d3d4a6889265e4e08063522 (127.13.38.1:39053): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97ddfabfb8b6471a8d086d268cd60119: Network error: Client connection negotiation failed: client connection to 127.13.38.1:39053: connect: Connection refused (error 111)
I20260812 17:52:28.372623 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 10298
I20260812 17:52:28.387936 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 9716
2026-08-12T17:52:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2804 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T17:52:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:28Z Disabled control of system clock
I20260812 17:52:28.431432 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:44709
--webserver_interface=127.13.38.62
--webserver_port=0
--builtin_ntp_servers=127.13.38.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:44709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 17:52:28.550417 10548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:28.550755 10548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:28.550794 10548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:28.555124 10548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:28.555218 10548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:28.555234 10548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:28.555253 10548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:28.555269 10548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:28.560503 10548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:44709
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:28.561815 10548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:28.563319 10548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:28.572983 10557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:28.572983 10558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:28.573004 10560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:28.573333 10548 server_base.cc:1034] running on GCE node
I20260812 17:52:28.574150 10548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:28.575513 10548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:28.576788 10548 hybrid_clock.cc:648] HybridClock initialized: now 1786557148576763 us; error 39 us; skew 500 ppm
I20260812 17:52:28.579308 10548 webserver.cc:460] Webserver started at http://127.13.38.62:36105/ using document root <none> and password file <none>
I20260812 17:52:28.580065 10548 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:28.580175 10548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:28.580425 10548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:28.582712 10548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "5c6a91115baa4ebfb40a3e71569b3bdd"
format_stamp: "Formatted at 2026-08-12 17:52:28 on dist-test-slave-sb2z"
server_key: "fe0c3f4e6c4e00a3c9f162157c1caf3d"
server_key_iv: "7bc1b4893baed7bef7abcc134dc3b6d2"
server_key_version: "encryptionkey@0"
I20260812 17:52:28.583388 10548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "5c6a91115baa4ebfb40a3e71569b3bdd"
format_stamp: "Formatted at 2026-08-12 17:52:28 on dist-test-slave-sb2z"
server_key: "fe0c3f4e6c4e00a3c9f162157c1caf3d"
server_key_iv: "7bc1b4893baed7bef7abcc134dc3b6d2"
server_key_version: "encryptionkey@0"
I20260812 17:52:28.587738 10548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:52:28.590636 10570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:28.592319 10548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:28.592491 10548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "5c6a91115baa4ebfb40a3e71569b3bdd"
format_stamp: "Formatted at 2026-08-12 17:52:28 on dist-test-slave-sb2z"
server_key: "fe0c3f4e6c4e00a3c9f162157c1caf3d"
server_key_iv: "7bc1b4893baed7bef7abcc134dc3b6d2"
server_key_version: "encryptionkey@0"
I20260812 17:52:28.592633 10548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:28.620976 10548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:28.621778 10548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:28.621994 10548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:28.629628 10548 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:44709
I20260812 17:52:28.629637 10639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:44709 every 8 connection(s)
I20260812 17:52:28.630703 10548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:28.634295 10642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:28.638981 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 10548
I20260812 17:52:28.639101 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:28.639396 13464 external_mini_cluster.cc:1428] Setting key d426156446642a89e3db483f56368517
I20260812 17:52:28.641063 10642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd: Bootstrap starting.
I20260812 17:52:28.643585 10642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:28.644451 10642 log.cc:824] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:28.647074 10642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd: No bootstrap required, opened a new log
I20260812 17:52:28.650123 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.641181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.13.38.62:44709)
0812 17:52:28.641872 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:28.641889 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:28.641926 (+ 37us) server_negotiation.cc:407] Connection header received
0812 17:52:28.642603 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:28.642619 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:28.643035 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:28.643336 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:28.643836 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.644882 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.645940 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.646647 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.646831 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.647853 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:28.647859 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:28.647877 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:28.648416 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:28.648490 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:28.648517 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:28.648621 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:28.648797 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 17:52:28.649115 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":135,"threads_started":1}
I20260812 17:52:28.652036 10642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6a91115baa4ebfb40a3e71569b3bdd" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44709 } }
I20260812 17:52:28.652275 10642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:28.652311 10642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c6a91115baa4ebfb40a3e71569b3bdd, State: Initialized, Role: FOLLOWER
I20260812 17:52:28.652832 10642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6a91115baa4ebfb40a3e71569b3bdd" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44709 } }
I20260812 17:52:28.653000 10642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:28.653071 10642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:28.653174 10642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:28.654206 10642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6a91115baa4ebfb40a3e71569b3bdd" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44709 } }
I20260812 17:52:28.654578 10642 leader_election.cc:302] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c6a91115baa4ebfb40a3e71569b3bdd; no voters:
I20260812 17:52:28.654924 10642 leader_election.cc:288] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:28.655206 10648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:28.655556 10648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [term 1 LEADER]: Becoming Leader. State: Replica: 5c6a91115baa4ebfb40a3e71569b3bdd, State: Running, Role: LEADER
I20260812 17:52:28.655922 10648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6a91115baa4ebfb40a3e71569b3bdd" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44709 } }
I20260812 17:52:28.656368 10642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:28.657721 10652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c6a91115baa4ebfb40a3e71569b3bdd. Latest consensus state: current_term: 1 leader_uuid: "5c6a91115baa4ebfb40a3e71569b3bdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6a91115baa4ebfb40a3e71569b3bdd" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44709 } } }
I20260812 17:52:28.657855 10652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:28.658164 10651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c6a91115baa4ebfb40a3e71569b3bdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6a91115baa4ebfb40a3e71569b3bdd" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44709 } } }
I20260812 17:52:28.658279 10651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:28.658263 10664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:28.662108 10664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:28.669430 10664 catalog_manager.cc:1347] Generated new cluster ID: 2bd9e41f89474be797c87110299dd089
I20260812 17:52:28.669561 10664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:28.679207 10664 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:28.680557 10664 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:28.689386 10664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd: Generated new TSK 0
I20260812 17:52:28.690335 10664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:28.702702 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44709
--builtin_ntp_servers=127.13.38.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 17:52:28.832173 10679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:28.832485 10679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:28.832528 10679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:28.836727 10679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:28.836803 10679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:28.836863 10679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:52:28.842034 10679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:28.843209 10679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:28.844756 10679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:28.855218 10691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:28.855211 10687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:28.855213 10686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:28.855654 10679 server_base.cc:1034] running on GCE node
I20260812 17:52:28.856127 10679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:28.856909 10679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:28.858115 10679 hybrid_clock.cc:648] HybridClock initialized: now 1786557148858079 us; error 48 us; skew 500 ppm
I20260812 17:52:28.860448 10679 webserver.cc:460] Webserver started at http://127.13.38.1:36831/ using document root <none> and password file <none>
I20260812 17:52:28.861158 10679 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:28.861272 10679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:28.861559 10679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:28.864043 10679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "20e4c5c1c8d34f2d9caf06bc202961cd"
format_stamp: "Formatted at 2026-08-12 17:52:28 on dist-test-slave-sb2z"
server_key: "997b48d893cea06c7ff09ae553b737c6"
server_key_iv: "0cbd284c850213e63d36356ac375b50f"
server_key_version: "encryptionkey@0"
I20260812 17:52:28.864737 10679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "20e4c5c1c8d34f2d9caf06bc202961cd"
format_stamp: "Formatted at 2026-08-12 17:52:28 on dist-test-slave-sb2z"
server_key: "997b48d893cea06c7ff09ae553b737c6"
server_key_iv: "0cbd284c850213e63d36356ac375b50f"
server_key_version: "encryptionkey@0"
I20260812 17:52:28.869344 10679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:52:28.872313 10699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:28.873642 10679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:28.873805 10679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "20e4c5c1c8d34f2d9caf06bc202961cd"
format_stamp: "Formatted at 2026-08-12 17:52:28 on dist-test-slave-sb2z"
server_key: "997b48d893cea06c7ff09ae553b737c6"
server_key_iv: "0cbd284c850213e63d36356ac375b50f"
server_key_version: "encryptionkey@0"
I20260812 17:52:28.873947 10679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:28.895949 10679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:28.897096 10679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:28.897341 10679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:28.898505 10679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:28.899580 10679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:28.899633 10679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:28.899677 10679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:28.899693 10679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:28.909101 10679 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:45543
I20260812 17:52:28.909126 10856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:45543 every 8 connection(s)
I20260812 17:52:28.910094 10679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:52:28.911567 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 10679
I20260812 17:52:28.911705 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:52:28.912138 13464 external_mini_cluster.cc:1428] Setting key b35162f2b9e48a4655dab0cf799d1dec
I20260812 17:52:28.914327 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44709
--builtin_ntp_servers=127.13.38.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 17:52:28.920755 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.912483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:56481 (local address 127.13.38.62:44709)
0812 17:52:28.912644 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:28.912649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:28.913946 (+ 1297us) server_negotiation.cc:407] Connection header received
0812 17:52:28.915570 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:28.915573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:28.915633 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:28.915747 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:28.916709 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.917267 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.918465 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.918694 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.918812 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.920222 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:28.920225 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:28.920227 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:28.920242 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:28.920262 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:28.920270 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:28.920355 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:28.920530 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 17:52:28.920600 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 17:52:28.921963 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:28.912971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44709 (local address 127.13.38.1:56481)
0812 17:52:28.913742 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:28.913785 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:28.915300 (+ 1515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:28.915941 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:28.915948 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:28.916049 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:28.916562 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.916575 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.917413 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.917416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.918320 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:28.918330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:28.918865 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:28.918869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:28.919049 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:28.919909 (+ 860us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:52:28.919911 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:28.919956 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:52:28.920031 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:28.920357 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:28.920371 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:28.920379 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 17:52:28.920527 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 17:52:28.920864 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":137,"threads_started":1}
I20260812 17:52:28.923621 10857 heartbeater.cc:347] Connected to a master server at 127.13.38.62:44709
I20260812 17:52:28.923914 10857 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:28.924562 10857 heartbeater.cc:511] Master 127.13.38.62:44709 requested a full tablet report, sending...
I20260812 17:52:28.926370 10593 ts_manager.cc:194] Registered new tserver with Master: 20e4c5c1c8d34f2d9caf06bc202961cd (127.13.38.1:45543)
I20260812 17:52:28.927700 10593 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.38.1:56481
W20260812 17:52:29.037500 10862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:29.037869 10862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:29.037957 10862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:29.042384 10862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:29.042488 10862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:29.042586 10862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:52:29.048105 10862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:29.049437 10862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:29.051082 10862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:29.062081 10874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:29.062173 10877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:29.062606 10873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:29.062714 10862 server_base.cc:1034] running on GCE node
I20260812 17:52:29.063340 10862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:29.064257 10862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:29.067672 10862 hybrid_clock.cc:648] HybridClock initialized: now 1786557149067582 us; error 88 us; skew 500 ppm
I20260812 17:52:29.070063 10862 webserver.cc:460] Webserver started at http://127.13.38.2:38139/ using document root <none> and password file <none>
I20260812 17:52:29.070742 10862 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:29.070820 10862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:29.071076 10862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:29.073551 10862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "eea5bc4190ce479f9d7e573aeb99a6dc"
format_stamp: "Formatted at 2026-08-12 17:52:29 on dist-test-slave-sb2z"
server_key: "ae15d290dd5bd700a2c4577ee7a13731"
server_key_iv: "9c78ecb2b57e8295379b1fda7c69c10e"
server_key_version: "encryptionkey@0"
I20260812 17:52:29.074200 10862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "eea5bc4190ce479f9d7e573aeb99a6dc"
format_stamp: "Formatted at 2026-08-12 17:52:29 on dist-test-slave-sb2z"
server_key: "ae15d290dd5bd700a2c4577ee7a13731"
server_key_iv: "9c78ecb2b57e8295379b1fda7c69c10e"
server_key_version: "encryptionkey@0"
I20260812 17:52:29.078681 10862 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 17:52:29.081755 10883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:29.083479 10862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:52:29.083699 10862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "eea5bc4190ce479f9d7e573aeb99a6dc"
format_stamp: "Formatted at 2026-08-12 17:52:29 on dist-test-slave-sb2z"
server_key: "ae15d290dd5bd700a2c4577ee7a13731"
server_key_iv: "9c78ecb2b57e8295379b1fda7c69c10e"
server_key_version: "encryptionkey@0"
I20260812 17:52:29.083874 10862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:29.114854 10862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:29.115733 10862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:29.115967 10862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:29.116624 10862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:29.117666 10862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:29.117754 10862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:29.117848 10862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:29.117900 10862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:29.127848 10862 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:38915
I20260812 17:52:29.127871 11044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:38915 every 8 connection(s)
I20260812 17:52:29.128906 10862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:52:29.133446 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 10862
I20260812 17:52:29.133584 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:52:29.133911 13464 external_mini_cluster.cc:1428] Setting key 843ff8baf771fd2a88ee7d54cd8b1d1b
I20260812 17:52:29.136189 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44709
--builtin_ntp_servers=127.13.38.20:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 17:52:29.139782 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.130942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:37323 (local address 127.13.38.62:44709)
0812 17:52:29.131095 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:29.131099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:29.132312 (+ 1213us) server_negotiation.cc:407] Connection header received
0812 17:52:29.134023 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:29.134027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:29.134096 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:29.134216 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:29.135479 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.136124 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.137496 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.137749 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.137880 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.139162 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:29.139166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:29.139168 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:29.139183 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:29.139203 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:29.139212 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:29.139303 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:29.139553 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 17:52:29.139625 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:52:29.140789 11048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.131301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44709 (local address 127.13.38.2:37323)
0812 17:52:29.132130 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:29.132177 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:29.133764 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:29.134438 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:29.134445 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:29.134543 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:29.135313 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.135333 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.136274 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.136280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.137339 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.137350 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.137940 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.137945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.138144 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.138845 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:52:29.138847 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:29.138892 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:52:29.138970 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:29.139306 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:29.139319 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:29.139328 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 17:52:29.139516 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 17:52:29.139876 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":189,"threads_started":1}
I20260812 17:52:29.142259 11046 heartbeater.cc:347] Connected to a master server at 127.13.38.62:44709
I20260812 17:52:29.142572 11046 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:29.143183 11046 heartbeater.cc:511] Master 127.13.38.62:44709 requested a full tablet report, sending...
I20260812 17:52:29.144532 10593 ts_manager.cc:194] Registered new tserver with Master: eea5bc4190ce479f9d7e573aeb99a6dc (127.13.38.2:38915)
I20260812 17:52:29.145198 10593 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.38.2:37323
W20260812 17:52:29.266520 11049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:29.266876 11049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:29.266968 11049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:29.271760 11049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:29.271850 11049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:29.271950 11049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:52:29.277753 11049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:29.279016 11049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:29.280542 11049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:29.291287 11054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:29.291292 11060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:29.291419 11056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:29.291896 11049 server_base.cc:1034] running on GCE node
I20260812 17:52:29.292371 11049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:29.293095 11049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:29.294305 11049 hybrid_clock.cc:648] HybridClock initialized: now 1786557149294240 us; error 79 us; skew 500 ppm
I20260812 17:52:29.296588 11049 webserver.cc:460] Webserver started at http://127.13.38.3:46725/ using document root <none> and password file <none>
I20260812 17:52:29.297256 11049 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:29.297326 11049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:29.297559 11049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:29.299885 11049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "5a89f039d6f141c8bf159bc82b202447"
format_stamp: "Formatted at 2026-08-12 17:52:29 on dist-test-slave-sb2z"
server_key: "f2775b579c248fdde793e540f6f520bc"
server_key_iv: "ac92c0ebc057ec5cc1b6a3e040122bc0"
server_key_version: "encryptionkey@0"
I20260812 17:52:29.300493 11049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "5a89f039d6f141c8bf159bc82b202447"
format_stamp: "Formatted at 2026-08-12 17:52:29 on dist-test-slave-sb2z"
server_key: "f2775b579c248fdde793e540f6f520bc"
server_key_iv: "ac92c0ebc057ec5cc1b6a3e040122bc0"
server_key_version: "encryptionkey@0"
I20260812 17:52:29.304832 11049 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:52:29.307885 11067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:29.309418 11049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:52:29.309610 11049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "5a89f039d6f141c8bf159bc82b202447"
format_stamp: "Formatted at 2026-08-12 17:52:29 on dist-test-slave-sb2z"
server_key: "f2775b579c248fdde793e540f6f520bc"
server_key_iv: "ac92c0ebc057ec5cc1b6a3e040122bc0"
server_key_version: "encryptionkey@0"
I20260812 17:52:29.309763 11049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:29.327174 11049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:29.328279 11049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:29.328516 11049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:29.329195 11049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:29.330281 11049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:29.330360 11049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:29.330432 11049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:29.330487 11049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:29.340497 11049 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:37247
I20260812 17:52:29.340515 11232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:37247 every 8 connection(s)
I20260812 17:52:29.341563 11049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:52:29.345829 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 11049
I20260812 17:52:29.345950 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:52:29.346288 13464 external_mini_cluster.cc:1428] Setting key d85d717db60ea5f7cdb9cf6adcdf0a96
I20260812 17:52:29.352329 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.343712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:54797 (local address 127.13.38.62:44709)
0812 17:52:29.343885 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:29.343889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:29.345077 (+ 1188us) server_negotiation.cc:407] Connection header received
0812 17:52:29.347041 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:29.347044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:29.347107 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:29.347222 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:29.348342 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.348942 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.350154 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.350381 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.350497 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.351780 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:29.351784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:29.351787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:29.351808 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:29.351827 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:29.351835 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:29.351916 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:29.352104 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 17:52:29.352172 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 17:52:29.353272 11238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.344103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:44709 (local address 127.13.38.3:54797)
0812 17:52:29.344853 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:29.344942 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:29.346792 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:29.347426 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:29.347434 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:29.347587 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:29.348190 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.348205 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.349085 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.349088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.350012 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.350023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.350555 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.350560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.350733 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.351421 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:52:29.351422 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:29.351465 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:52:29.351611 (+ 146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:29.351923 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:29.351935 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:29.351941 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 17:52:29.352090 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 17:52:29.352422 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":134,"threads_started":1}
I20260812 17:52:29.354583 11234 heartbeater.cc:347] Connected to a master server at 127.13.38.62:44709
I20260812 17:52:29.354879 11234 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:29.355607 11234 heartbeater.cc:511] Master 127.13.38.62:44709 requested a full tablet report, sending...
I20260812 17:52:29.356859 10593 ts_manager.cc:194] Registered new tserver with Master: 5a89f039d6f141c8bf159bc82b202447 (127.13.38.3:37247)
I20260812 17:52:29.357834 10593 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.38.3:54797
I20260812 17:52:29.361117 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:52:29.367684 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.363452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.13.38.62:44709)
0812 17:52:29.363651 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:29.363656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:29.363799 (+ 143us) server_negotiation.cc:407] Connection header received
0812 17:52:29.364010 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:29.364013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:29.364055 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:29.364145 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:29.364486 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.365026 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.365898 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.366120 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.366227 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.367126 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:29.367130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:29.367132 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:29.367146 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:29.367163 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:29.367171 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:29.367302 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:29.367452 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 17:52:29.367492 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260812 17:52:29.371109 10593 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:52:29.374496 10593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:52:29.391014 11255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.384774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:38915 (local address 127.0.0.1:58354)
0812 17:52:29.385833 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:29.385847 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:29.385991 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:29.386710 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:29.386716 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:29.386738 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:29.386830 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.386835 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.388394 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.388397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.389321 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.389355 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.390417 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.390421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.390531 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.390585 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 17:52:29.390703 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 17:52:29.390838 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260812 17:52:29.391211 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.385316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58354 (local address 127.13.38.2:38915)
0812 17:52:29.385690 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:29.385703 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:29.385922 (+ 219us) server_negotiation.cc:407] Connection header received
0812 17:52:29.386170 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:29.386176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:29.386403 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:29.386536 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:29.386968 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.388280 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.389487 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.390285 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.390396 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.390607 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:29.390755 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 17:52:29.390880 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":159,"threads_started":1}
I20260812 17:52:29.391322 11256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.384421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45543 (local address 127.0.0.1:49908)
0812 17:52:29.385393 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:29.385449 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:29.385592 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:29.386388 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:29.386395 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:29.386417 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:29.386575 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.386583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.388353 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.388356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.389512 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.389521 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.390881 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.390885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.390983 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.391010 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 17:52:29.391071 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 17:52:29.391147 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"mutex_wait_us":34,"thread_start_us":134,"threads_started":1}
I20260812 17:52:29.391453 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.385206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53310 (local address 127.13.38.3:37247)
0812 17:52:29.385912 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:29.385924 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:29.385946 (+ 22us) server_negotiation.cc:407] Connection header received
0812 17:52:29.386013 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:29.386017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:29.386202 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:29.386366 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:29.386759 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.387678 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.389413 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.390242 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.390439 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.390640 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:29.391160 (+ 520us) server_negotiation.cc:299] Negotiation successful
0812 17:52:29.391275 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":119,"threads_started":1}
I20260812 17:52:29.391059 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.384475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:37247 (local address 127.0.0.1:53310)
0812 17:52:29.385691 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:29.385707 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:29.385807 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:29.386527 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:29.386532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:29.386546 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:29.386631 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.386638 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.387831 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.387835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.389267 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.389279 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.390707 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.390712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.390798 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.390819 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 17:52:29.390885 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 17:52:29.390944 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1152,"mutex_wait_us":103,"thread_start_us":130,"threads_started":1}
I20260812 17:52:29.392483 11257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.384731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49908 (local address 127.13.38.1:45543)
0812 17:52:29.385481 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:29.385491 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:29.385525 (+ 34us) server_negotiation.cc:407] Connection header received
0812 17:52:29.385976 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:29.385982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:29.386202 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:29.386736 (+ 534us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:29.386867 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.388219 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.389658 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.390743 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.391768 (+ 1025us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.391961 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:29.392011 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 17:52:29.392108 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":64,"threads_started":1}
I20260812 17:52:29.394661 10955 tablet_service.cc:1467] Processing CreateTablet for tablet 81db0287e06f4f92b64497c738b3240a (DEFAULT_TABLE table=test-table [id=6a8a17958c7a43518a0b20126d5ac26d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:29.394657 11146 tablet_service.cc:1467] Processing CreateTablet for tablet 81db0287e06f4f92b64497c738b3240a (DEFAULT_TABLE table=test-table [id=6a8a17958c7a43518a0b20126d5ac26d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:29.394881 10767 tablet_service.cc:1467] Processing CreateTablet for tablet 81db0287e06f4f92b64497c738b3240a (DEFAULT_TABLE table=test-table [id=6a8a17958c7a43518a0b20126d5ac26d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:29.395957 10767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81db0287e06f4f92b64497c738b3240a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:29.395957 11146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81db0287e06f4f92b64497c738b3240a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:29.395970 10955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81db0287e06f4f92b64497c738b3240a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:29.402951 11263 tablet_bootstrap.cc:492] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd: Bootstrap starting.
I20260812 17:52:29.404454 11264 tablet_bootstrap.cc:492] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc: Bootstrap starting.
I20260812 17:52:29.404454 11265 tablet_bootstrap.cc:492] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447: Bootstrap starting.
I20260812 17:52:29.406342 11263 tablet_bootstrap.cc:654] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:29.406710 11265 tablet_bootstrap.cc:654] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:29.406710 11264 tablet_bootstrap.cc:654] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:29.407204 11263 log.cc:824] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:29.407603 11265 log.cc:824] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:29.407604 11264 log.cc:824] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:29.409565 11263 tablet_bootstrap.cc:492] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd: No bootstrap required, opened a new log
I20260812 17:52:29.409569 11264 tablet_bootstrap.cc:492] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc: No bootstrap required, opened a new log
I20260812 17:52:29.409711 11265 tablet_bootstrap.cc:492] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447: No bootstrap required, opened a new log
I20260812 17:52:29.409783 11264 ts_tablet_manager.cc:1397] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 17:52:29.409783 11263 ts_tablet_manager.cc:1397] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 17:52:29.409935 11265 ts_tablet_manager.cc:1397] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:52:29.413813 11265 raft_consensus.cc:348] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45543 } } peers { permanent_uuid: "5a89f039d6f141c8bf159bc82b202447" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37247 } } peers { permanent_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 38915 } }
I20260812 17:52:29.414143 11265 raft_consensus.cc:374] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:29.414214 11265 raft_consensus.cc:729] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a89f039d6f141c8bf159bc82b202447, State: Initialized, Role: FOLLOWER
I20260812 17:52:29.414410 11264 raft_consensus.cc:348] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45543 } } peers { permanent_uuid: "5a89f039d6f141c8bf159bc82b202447" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37247 } } peers { permanent_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 38915 } }
I20260812 17:52:29.414410 11263 raft_consensus.cc:348] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45543 } } peers { permanent_uuid: "5a89f039d6f141c8bf159bc82b202447" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37247 } } peers { permanent_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 38915 } }
I20260812 17:52:29.414676 11264 raft_consensus.cc:374] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:29.414676 11263 raft_consensus.cc:374] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:29.414737 11263 raft_consensus.cc:729] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20e4c5c1c8d34f2d9caf06bc202961cd, State: Initialized, Role: FOLLOWER
I20260812 17:52:29.414737 11264 raft_consensus.cc:729] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eea5bc4190ce479f9d7e573aeb99a6dc, State: Initialized, Role: FOLLOWER
I20260812 17:52:29.414746 11265 consensus_queue.cc:260] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45543 } } peers { permanent_uuid: "5a89f039d6f141c8bf159bc82b202447" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37247 } } peers { permanent_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 38915 } }
I20260812 17:52:29.415239 11264 consensus_queue.cc:260] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45543 } } peers { permanent_uuid: "5a89f039d6f141c8bf159bc82b202447" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37247 } } peers { permanent_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 38915 } }
I20260812 17:52:29.415239 11263 consensus_queue.cc:260] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45543 } } peers { permanent_uuid: "5a89f039d6f141c8bf159bc82b202447" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37247 } } peers { permanent_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 38915 } }
I20260812 17:52:29.415628 11234 heartbeater.cc:503] Master 127.13.38.62:44709 was elected leader, sending a full tablet report...
I20260812 17:52:29.415982 11265 ts_tablet_manager.cc:1428] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260812 17:52:29.416285 11264 ts_tablet_manager.cc:1428] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260812 17:52:29.416285 11263 ts_tablet_manager.cc:1428] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260812 17:52:29.416678 10857 heartbeater.cc:503] Master 127.13.38.62:44709 was elected leader, sending a full tablet report...
I20260812 17:52:29.417706 11046 heartbeater.cc:503] Master 127.13.38.62:44709 was elected leader, sending a full tablet report...
I20260812 17:52:29.464170 11274 raft_consensus.cc:482] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:52:29.464454 11274 raft_consensus.cc:504] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45543 } } peers { permanent_uuid: "5a89f039d6f141c8bf159bc82b202447" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37247 } } peers { permanent_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 38915 } }
I20260812 17:52:29.465778 11274 leader_election.cc:288] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20e4c5c1c8d34f2d9caf06bc202961cd (127.13.38.1:45543), eea5bc4190ce479f9d7e573aeb99a6dc (127.13.38.2:38915)
I20260812 17:52:29.470109 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.466229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:47811 (local address 127.13.38.2:38915)
0812 17:52:29.466385 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:29.466391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:29.466455 (+ 64us) server_negotiation.cc:407] Connection header received
0812 17:52:29.466528 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:29.466535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:29.466608 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:29.466715 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:29.467258 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.467938 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.469124 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.469519 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.469646 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.469711 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:29.469901 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 17:52:29.469974 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 17:52:29.470110 11238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.466106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:38915 (local address 127.13.38.3:47811)
0812 17:52:29.466230 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:29.466252 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:29.466364 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:29.466736 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:29.466740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:29.466795 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:29.466972 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.466982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.468068 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.468075 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.468979 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.468989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.469653 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.469656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.469760 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.469790 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 17:52:29.469882 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 17:52:29.469961 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"spinlock_wait_cycles":13952}
I20260812 17:52:29.470243 11257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.466352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:51591 (local address 127.13.38.1:45543)
0812 17:52:29.466509 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:29.466514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:29.466745 (+ 231us) server_negotiation.cc:407] Connection header received
0812 17:52:29.466922 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:29.466930 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:29.466977 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:29.467149 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:29.467443 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.468159 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.469236 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.469644 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.469771 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.469831 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:29.470037 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 17:52:29.470114 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 17:52:29.470302 11277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.466112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45543 (local address 127.13.38.3:51591)
0812 17:52:29.466638 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:29.466651 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:29.466809 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:29.467156 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:29.467159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:29.467177 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:29.467314 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.467320 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.468261 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.468265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.469102 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:29.469111 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.469785 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.469788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:29.469901 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.469920 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 17:52:29.470019 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 17:52:29.470090 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":134,"spinlock_wait_cycles":34432,"thread_start_us":155,"threads_started":1}
I20260812 17:52:29.471010 10801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81db0287e06f4f92b64497c738b3240a" candidate_uuid: "5a89f039d6f141c8bf159bc82b202447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd" is_pre_election: true
I20260812 17:52:29.471011 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81db0287e06f4f92b64497c738b3240a" candidate_uuid: "5a89f039d6f141c8bf159bc82b202447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc" is_pre_election: true
I20260812 17:52:29.471370 10801 raft_consensus.cc:2454] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a89f039d6f141c8bf159bc82b202447 in term 0.
I20260812 17:52:29.471370 10982 raft_consensus.cc:2454] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a89f039d6f141c8bf159bc82b202447 in term 0.
I20260812 17:52:29.471987 11071 leader_election.cc:302] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 20e4c5c1c8d34f2d9caf06bc202961cd, 5a89f039d6f141c8bf159bc82b202447, eea5bc4190ce479f9d7e573aeb99a6dc; no voters:
I20260812 17:52:29.472304 11274 raft_consensus.cc:2781] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:52:29.472396 11274 raft_consensus.cc:482] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:52:29.472440 11274 raft_consensus.cc:3037] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:29.473676 11274 raft_consensus.cc:504] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45543 } } peers { permanent_uuid: "5a89f039d6f141c8bf159bc82b202447" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37247 } } peers { permanent_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 38915 } }
I20260812 17:52:29.474072 11274 leader_election.cc:288] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [CANDIDATE]: Term 1 election: Requested vote from peers 20e4c5c1c8d34f2d9caf06bc202961cd (127.13.38.1:45543), eea5bc4190ce479f9d7e573aeb99a6dc (127.13.38.2:38915)
I20260812 17:52:29.474558 10801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81db0287e06f4f92b64497c738b3240a" candidate_uuid: "5a89f039d6f141c8bf159bc82b202447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd"
I20260812 17:52:29.474706 10801 raft_consensus.cc:3037] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:29.474653 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81db0287e06f4f92b64497c738b3240a" candidate_uuid: "5a89f039d6f141c8bf159bc82b202447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc"
I20260812 17:52:29.474846 10982 raft_consensus.cc:3037] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:29.475898 10801 raft_consensus.cc:2454] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a89f039d6f141c8bf159bc82b202447 in term 1.
I20260812 17:52:29.476037 10982 raft_consensus.cc:2454] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a89f039d6f141c8bf159bc82b202447 in term 1.
I20260812 17:52:29.476374 11071 leader_election.cc:302] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20e4c5c1c8d34f2d9caf06bc202961cd, 5a89f039d6f141c8bf159bc82b202447; no voters:
I20260812 17:52:29.476631 11274 raft_consensus.cc:2781] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:29.476873 11274 raft_consensus.cc:686] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 1 LEADER]: Becoming Leader. State: Replica: 5a89f039d6f141c8bf159bc82b202447, State: Running, Role: LEADER
I20260812 17:52:29.477283 11274 consensus_queue.cc:237] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45543 } } peers { permanent_uuid: "5a89f039d6f141c8bf159bc82b202447" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37247 } } peers { permanent_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 38915 } }
I20260812 17:52:29.480906 10592 catalog_manager.cc:5495] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a89f039d6f141c8bf159bc82b202447 (127.13.38.3). New cstate: current_term: 1 leader_uuid: "5a89f039d6f141c8bf159bc82b202447" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a89f039d6f141c8bf159bc82b202447" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 37247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 38915 } health_report { overall_health: UNKNOWN } } }
I20260812 17:52:29.497890 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.493719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.13.38.3:37247)
0812 17:52:29.493875 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:29.493878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:29.493893 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:52:29.494026 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:29.494028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:29.494090 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:29.494187 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:52:29.494561 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.495103 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.496149 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:29.496368 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:29.496464 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:29.496872 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:52:29.497329 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:52:29.497449 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:29.497735 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 17:52:29.497773 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 17:52:29.504626 10801 raft_consensus.cc:1261] T 81db0287e06f4f92b64497c738b3240a P 20e4c5c1c8d34f2d9caf06bc202961cd [term 1 FOLLOWER]: Refusing update from remote peer 5a89f039d6f141c8bf159bc82b202447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:52:29.504626 10982 raft_consensus.cc:1261] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc [term 1 FOLLOWER]: Refusing update from remote peer 5a89f039d6f141c8bf159bc82b202447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:52:29.505406 11274 consensus_queue.cc:1035] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eea5bc4190ce479f9d7e573aeb99a6dc" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 38915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:52:29.505643 11279 consensus_queue.cc:1035] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20e4c5c1c8d34f2d9caf06bc202961cd" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:52:29.514645 11284 mvcc.cc:205] Tried to move back new op lower bound from 7317738084362190848 to 7317738084262113280. Current Snapshot: MvccSnapshot[applied={T|T < 7317738084362190848}]
I20260812 17:52:29.514963 11283 mvcc.cc:205] Tried to move back new op lower bound from 7317738084362190848 to 7317738084262113280. Current Snapshot: MvccSnapshot[applied={T|T < 7317738084362190848}]
I20260812 17:52:29.524190 10592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:52:29.524402 10592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:52:29.527117 10592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c6a91115baa4ebfb40a3e71569b3bdd: Sending DeleteTablet for 3 replicas of tablet 81db0287e06f4f92b64497c738b3240a
I20260812 17:52:29.528118 10767 tablet_service.cc:1514] Processing DeleteTablet for tablet 81db0287e06f4f92b64497c738b3240a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:29 UTC) from {username='slave'} at 127.0.0.1:49908
I20260812 17:52:29.528347 11146 tablet_service.cc:1514] Processing DeleteTablet for tablet 81db0287e06f4f92b64497c738b3240a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:29 UTC) from {username='slave'} at 127.0.0.1:53310
I20260812 17:52:29.528784 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 10679
I20260812 17:52:29.564513 11296 tablet_replica.cc:330] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447: stopping tablet replica
I20260812 17:52:29.564790 10955 tablet_service.cc:1514] Processing DeleteTablet for tablet 81db0287e06f4f92b64497c738b3240a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:29 UTC) from {username='slave'} at 127.0.0.1:58354
I20260812 17:52:29.565025 11296 raft_consensus.cc:2229] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:52:29.565522 11296 raft_consensus.cc:2258] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:52:29.565570 11297 tablet_replica.cc:330] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc: stopping tablet replica
I20260812 17:52:29.566001 11297 raft_consensus.cc:2229] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:52:29.566392 11297 raft_consensus.cc:2258] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:52:29.567799 11296 ts_tablet_manager.cc:1905] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:52:29.568105 11297 ts_tablet_manager.cc:1905] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 17:52:29.571416 10574 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:45543 (error 108)
I20260812 17:52:29.571720 11297 ts_tablet_manager.cc:1918] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:52:29.571934 11297 log.cc:1196] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/81db0287e06f4f92b64497c738b3240a
I20260812 17:52:29.571949 11296 ts_tablet_manager.cc:1918] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:52:29.572068 11296 log.cc:1196] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/81db0287e06f4f92b64497c738b3240a
I20260812 17:52:29.572461 11296 ts_tablet_manager.cc:1939] T 81db0287e06f4f92b64497c738b3240a P 5a89f039d6f141c8bf159bc82b202447: Deleting consensus metadata
I20260812 17:52:29.572497 11297 ts_tablet_manager.cc:1939] T 81db0287e06f4f92b64497c738b3240a P eea5bc4190ce479f9d7e573aeb99a6dc: Deleting consensus metadata
I20260812 17:52:29.572602 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 10862
I20260812 17:52:29.593720 10571 catalog_manager.cc:4841] TS 5a89f039d6f141c8bf159bc82b202447 (127.13.38.3:37247): tablet 81db0287e06f4f92b64497c738b3240a (table test-table [id=6a8a17958c7a43518a0b20126d5ac26d]) successfully deleted
I20260812 17:52:29.603513 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 11049
I20260812 17:52:29.603937 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.603565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:45543 (local address 127.0.0.1:49914)
0812 17:52:29.603712 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:29.603817 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:45543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 17:52:29.613752 10574 catalog_manager.cc:4584] TS 20e4c5c1c8d34f2d9caf06bc202961cd (127.13.38.1:45543): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 81db0287e06f4f92b64497c738b3240a: Network error: Client connection negotiation failed: client connection to 127.13.38.1:45543: connect: Connection refused (error 111)
I20260812 17:52:29.616735 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:29.616415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:38915 (local address 127.0.0.1:58360)
0812 17:52:29.616561 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:29.616632 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.2:38915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 17:52:29.622750 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 10548
2026-08-12T17:52:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1232 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:29 dist-test-slave-sb2z krb5kdc[11301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:29 dist-test-slave-sb2z krb5kdc[11301](info): set up 2 sockets
Aug 12 17:52:29 dist-test-slave-sb2z krb5kdc[11301](info): commencing operation
krb5kdc: starting...
W20260812 17:52:30.934402 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:31 dist-test-slave-sb2z krb5kdc[11301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:52:31.088027 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:33717
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:33717
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:31.213222 11316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:31.213593 11316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:31.213681 11316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:31.218027 11316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:31.218146 11316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:31.218184 11316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:31.218227 11316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:31.218264 11316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:31.224149 11316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:33717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:33717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:31.225425 11316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:31.226559 11316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:31.236595 11325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:31.236611 11330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:31.236784 11316 server_base.cc:1034] running on GCE node
W20260812 17:52:31.237044 11326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:31.237730 11316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:31.239078 11316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:31.240324 11316 hybrid_clock.cc:648] HybridClock initialized: now 1786557151240274 us; error 69 us; skew 500 ppm
Aug 12 17:52:31 dist-test-slave-sb2z krb5kdc[11301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557151, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:31.244004 11316 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:52:31.245450 11316 webserver.cc:460] Webserver started at http://127.13.38.62:44501/ using document root <none> and password file <none>
I20260812 17:52:31.246464 11316 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:31.246726 11316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:31.247066 11316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:31.249557 11316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "d3dd8ea01129457eacf516e66ab70a77"
format_stamp: "Formatted at 2026-08-12 17:52:31 on dist-test-slave-sb2z"
server_key: "2944c7dca4523b82393075ce047b7e27"
server_key_iv: "b82d2382034a65420c3ed13bad070de2"
server_key_version: "encryptionkey@0"
I20260812 17:52:31.250232 11316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "d3dd8ea01129457eacf516e66ab70a77"
format_stamp: "Formatted at 2026-08-12 17:52:31 on dist-test-slave-sb2z"
server_key: "2944c7dca4523b82393075ce047b7e27"
server_key_iv: "b82d2382034a65420c3ed13bad070de2"
server_key_version: "encryptionkey@0"
I20260812 17:52:31.254928 11316 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 17:52:31.259047 11337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:31.260815 11316 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260812 17:52:31.261058 11316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "d3dd8ea01129457eacf516e66ab70a77"
format_stamp: "Formatted at 2026-08-12 17:52:31 on dist-test-slave-sb2z"
server_key: "2944c7dca4523b82393075ce047b7e27"
server_key_iv: "b82d2382034a65420c3ed13bad070de2"
server_key_version: "encryptionkey@0"
I20260812 17:52:31.261443 11316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:31.283751 11316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:31.287732 11316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:31.287966 11316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:31.296424 11316 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:33717
I20260812 17:52:31.296463 11414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:33717 every 8 connection(s)
I20260812 17:52:31.297600 11316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:31.300863 11417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:31.307165 11417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77: Bootstrap starting.
I20260812 17:52:31.307754 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 11316
I20260812 17:52:31.307902 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:31.308192 13464 external_mini_cluster.cc:1428] Setting key 036eedf68e7811a8131a5fe42e51540d
I20260812 17:52:31.310386 11417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:31.311220 11417 log.cc:824] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:31.313460 11417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77: No bootstrap required, opened a new log
I20260812 17:52:31.317706 11417 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dd8ea01129457eacf516e66ab70a77" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33717 } }
I20260812 17:52:31.318106 11417 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:31.318219 11417 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3dd8ea01129457eacf516e66ab70a77, State: Initialized, Role: FOLLOWER
Aug 12 17:52:31 dist-test-slave-sb2z krb5kdc[11301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:52:31.318837 11417 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dd8ea01129457eacf516e66ab70a77" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33717 } }
I20260812 17:52:31.319056 11417 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:31.319144 11417 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:31.319267 11417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:31.320387 11417 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dd8ea01129457eacf516e66ab70a77" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33717 } }
I20260812 17:52:31.320815 11417 leader_election.cc:302] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3dd8ea01129457eacf516e66ab70a77; no voters:
I20260812 17:52:31.321264 11417 leader_election.cc:288] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:31.321557 11424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:31.321880 11424 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [term 1 LEADER]: Becoming Leader. State: Replica: d3dd8ea01129457eacf516e66ab70a77, State: Running, Role: LEADER
I20260812 17:52:31.322317 11424 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dd8ea01129457eacf516e66ab70a77" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33717 } }
I20260812 17:52:31.323001 11417 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:31.324020 11424 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3dd8ea01129457eacf516e66ab70a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dd8ea01129457eacf516e66ab70a77" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33717 } } }
I20260812 17:52:31.324230 11424 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:31.324493 11424 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3dd8ea01129457eacf516e66ab70a77. Latest consensus state: current_term: 1 leader_uuid: "d3dd8ea01129457eacf516e66ab70a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dd8ea01129457eacf516e66ab70a77" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33717 } } }
I20260812 17:52:31.324653 11424 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:31.325320 11433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:31.327708 11421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:31.309757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43968 (local address 127.13.38.62:33717)
0812 17:52:31.310357 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:31.310381 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:31.310422 (+ 41us) server_negotiation.cc:407] Connection header received
0812 17:52:31.311390 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:31.311419 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:31.311901 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:31.312298 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:31.313231 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:31.314329 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:31.315248 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:31.315843 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:31.316026 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:31.319705 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:31.319742 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:31.319767 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:31.319842 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:31.323036 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:31.323798 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:31.323806 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:31.323813 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:31.323979 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:31.324351 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:31.324357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:31.324361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:31.325073 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:31.325318 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:31.326246 (+ 928us) server_negotiation.cc:299] Negotiation successful
0812 17:52:31.326598 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":167,"threads_started":1}
I20260812 17:52:31.329257 11433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:31.336742 11433 catalog_manager.cc:1347] Generated new cluster ID: 85889524115b45f680a9a7b20df1f901
I20260812 17:52:31.336872 11433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:31.350266 11433 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:31.351651 11433 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:31.362907 11433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77: Generated new TSK 0
I20260812 17:52:31.363933 11433 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.38.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
Entry for principal kudu/127.13.38.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab.
I20260812 17:52:31.413211 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:33717
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:31.538507 11451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:31.538888 11451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:31.538975 11451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:31.543329 11451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:31.543464 11451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:31.543596 11451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:52:31.549345 11451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.1.keytab
--principal=kudu/127.13.38.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:33717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:31.550664 11451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:31.551887 11451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:31.563745 11459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:31.563735 11460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:31.563755 11462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:31.564205 11451 server_base.cc:1034] running on GCE node
I20260812 17:52:31.564845 11451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:31.565670 11451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:31.566883 11451 hybrid_clock.cc:648] HybridClock initialized: now 1786557151566838 us; error 61 us; skew 500 ppm
Aug 12 17:52:31 dist-test-slave-sb2z krb5kdc[11301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557151, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:31.570531 11451 init.cc:377] Logged in from keytab as kudu/127.13.38.1@KRBTEST.COM (short username kudu)
I20260812 17:52:31.571712 11451 webserver.cc:460] Webserver started at http://127.13.38.1:46549/ using document root <none> and password file <none>
I20260812 17:52:31.572393 11451 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:31.572497 11451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:31.572758 11451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:31.575012 11451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "51e6878fe3ed44c39ec157f36a8d1c9f"
format_stamp: "Formatted at 2026-08-12 17:52:31 on dist-test-slave-sb2z"
server_key: "3c4fd077c3475f794695ce9520bbb7ef"
server_key_iv: "dff40110408f97a40410fa36bee0645f"
server_key_version: "encryptionkey@0"
I20260812 17:52:31.575739 11451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "51e6878fe3ed44c39ec157f36a8d1c9f"
format_stamp: "Formatted at 2026-08-12 17:52:31 on dist-test-slave-sb2z"
server_key: "3c4fd077c3475f794695ce9520bbb7ef"
server_key_iv: "dff40110408f97a40410fa36bee0645f"
server_key_version: "encryptionkey@0"
I20260812 17:52:31.580032 11451 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:52:31.583060 11474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:31.584614 11451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:31.584786 11451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "51e6878fe3ed44c39ec157f36a8d1c9f"
format_stamp: "Formatted at 2026-08-12 17:52:31 on dist-test-slave-sb2z"
server_key: "3c4fd077c3475f794695ce9520bbb7ef"
server_key_iv: "dff40110408f97a40410fa36bee0645f"
server_key_version: "encryptionkey@0"
I20260812 17:52:31.584977 11451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:31.604118 11451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:31.607846 11451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:31.608103 11451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:31.608788 11451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:31.609779 11451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:31.609858 11451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:31.609942 11451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:31.609988 11451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:31.620365 11451 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:33563
I20260812 17:52:31.620385 11638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:33563 every 8 connection(s)
I20260812 17:52:31.621446 11451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:52:31.622566 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 11451
I20260812 17:52:31.622720 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:52:31.623092 13464 external_mini_cluster.cc:1428] Setting key 1665fa5de96d75536cbfe4bf0a919dc5
Aug 12 17:52:31 dist-test-slave-sb2z krb5kdc[11301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557151, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.1@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.2@KRBTEST.COM" created.
I20260812 17:52:31.638796 11421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:31.623816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:43743 (local address 127.13.38.62:33717)
0812 17:52:31.623971 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:31.623975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:31.625129 (+ 1154us) server_negotiation.cc:407] Connection header received
0812 17:52:31.626579 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:31.626582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:31.626642 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:31.626727 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:31.628188 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:31.628750 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:31.630035 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:31.630297 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:31.630434 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:31.633709 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:31.633713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:31.633715 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:31.633746 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:31.635434 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:31.636298 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:31.636302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:31.636303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:31.636436 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:31.637109 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:31.637112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:31.637113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:31.637388 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:31.637480 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:31.638324 (+ 844us) server_negotiation.cc:299] Negotiation successful
0812 17:52:31.638510 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 17:52:31.640240 11643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:31.624308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:33717 (local address 127.13.38.1:43743)
0812 17:52:31.624885 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:31.624992 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:31.626321 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:31.626935 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:31.626943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:31.627369 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:31.628009 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:31.628028 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:31.628923 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:31.628927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:31.629871 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:31.629882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:31.630487 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:31.630491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:31.630685 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:31.631438 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:52:31.631459 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:31.633482 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:31.635634 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:31.635642 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:31.635664 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:31.636133 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:31.636602 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:31.636605 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:31.636606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:31.636907 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:31.637554 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:31.637562 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:31.637904 (+ 342us) client_negotiation.cc:769] Sending connection context
0812 17:52:31.638183 (+ 279us) client_negotiation.cc:240] Negotiation successful
0812 17:52:31.638660 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":168,"threads_started":1}
I20260812 17:52:31.641873 11641 heartbeater.cc:347] Connected to a master server at 127.13.38.62:33717
I20260812 17:52:31.642207 11641 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:31.642975 11641 heartbeater.cc:511] Master 127.13.38.62:33717 requested a full tablet report, sending...
I20260812 17:52:31.645371 11360 ts_manager.cc:194] Registered new tserver with Master: 51e6878fe3ed44c39ec157f36a8d1c9f (127.13.38.1:33563)
I20260812 17:52:31.647325 11360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.1@KRBTEST.COM'} at 127.13.38.1:43743
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
Entry for principal kudu/127.13.38.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab.
I20260812 17:52:31.664125 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:33717
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:31.795889 11651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:31.796216 11651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:31.796269 11651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:31.800671 11651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:31.800763 11651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:31.800830 11651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:52:31.806526 11651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.2.keytab
--principal=kudu/127.13.38.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:33717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:31.807816 11651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:31.808898 11651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:31.819415 11660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:31.819443 11658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:31.819415 11657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:31.819689 11651 server_base.cc:1034] running on GCE node
I20260812 17:52:31.820379 11651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:31.821152 11651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:31.822371 11651 hybrid_clock.cc:648] HybridClock initialized: now 1786557151822320 us; error 68 us; skew 500 ppm
Aug 12 17:52:31 dist-test-slave-sb2z krb5kdc[11301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557151, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:31.825804 11651 init.cc:377] Logged in from keytab as kudu/127.13.38.2@KRBTEST.COM (short username kudu)
I20260812 17:52:31.827100 11651 webserver.cc:460] Webserver started at http://127.13.38.2:35839/ using document root <none> and password file <none>
I20260812 17:52:31.827816 11651 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:31.827898 11651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:31.828154 11651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:31.830518 11651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "2f421b8849b74ba39e86c3b346b64fc5"
format_stamp: "Formatted at 2026-08-12 17:52:31 on dist-test-slave-sb2z"
server_key: "a55de4fac0c60c513954af68d5fda4aa"
server_key_iv: "0cac2f1068b137342728e1702632100e"
server_key_version: "encryptionkey@0"
I20260812 17:52:31.831171 11651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "2f421b8849b74ba39e86c3b346b64fc5"
format_stamp: "Formatted at 2026-08-12 17:52:31 on dist-test-slave-sb2z"
server_key: "a55de4fac0c60c513954af68d5fda4aa"
server_key_iv: "0cac2f1068b137342728e1702632100e"
server_key_version: "encryptionkey@0"
I20260812 17:52:31.835791 11651 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 17:52:31.839634 11671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:31.841403 11651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 17:52:31.841603 11651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "2f421b8849b74ba39e86c3b346b64fc5"
format_stamp: "Formatted at 2026-08-12 17:52:31 on dist-test-slave-sb2z"
server_key: "a55de4fac0c60c513954af68d5fda4aa"
server_key_iv: "0cac2f1068b137342728e1702632100e"
server_key_version: "encryptionkey@0"
I20260812 17:52:31.841770 11651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:31.854130 11651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:31.857851 11651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:31.858083 11651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:31.858757 11651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:31.859771 11651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:31.859825 11651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:31.859870 11651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:31.859925 11651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:31.870353 11651 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:40357
I20260812 17:52:31.870374 11838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:40357 every 8 connection(s)
I20260812 17:52:31.871357 11651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:52:31.873330 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 11651
I20260812 17:52:31.873485 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:52:31.873817 13464 external_mini_cluster.cc:1428] Setting key 8f77ced0eaec267b137e8542ffd78e80
Aug 12 17:52:31 dist-test-slave-sb2z krb5kdc[11301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557151, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.2@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.3@KRBTEST.COM; defaulting to no policy
I20260812 17:52:31.887202 11421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:31.873495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:47939 (local address 127.13.38.62:33717)
0812 17:52:31.873673 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:31.873678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:31.874701 (+ 1023us) server_negotiation.cc:407] Connection header received
0812 17:52:31.876236 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:31.876242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:31.876311 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:31.876416 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:31.877801 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:31.878347 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:31.879684 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:31.879914 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:31.880047 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:31.883142 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:31.883146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:31.883148 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:31.883176 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:31.884768 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:31.885438 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:31.885441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:31.885442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:31.885555 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:31.885950 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:31.885953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:31.885954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:31.886165 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:31.886250 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:31.886842 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 17:52:31.886980 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.3@KRBTEST.COM" created.
I20260812 17:52:31.888087 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:31.873987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:33717 (local address 127.13.38.2:47939)
0812 17:52:31.874525 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:31.874586 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:31.876001 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:31.876661 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:31.876669 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:31.877080 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:31.877619 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:31.877635 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:31.878534 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:31.878538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:31.879485 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:31.879495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:31.880095 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:31.880100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:31.880298 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:31.880988 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:52:31.881006 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:31.882922 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:31.884940 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:31.884950 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:31.884976 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:31.885321 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:31.885655 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:31.885657 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:31.885659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:31.885812 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:31.886271 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:31.886277 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:31.886552 (+ 275us) client_negotiation.cc:769] Sending connection context
0812 17:52:31.886748 (+ 196us) client_negotiation.cc:240] Negotiation successful
0812 17:52:31.887042 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":164,"threads_started":1}
I20260812 17:52:31.889587 11839 heartbeater.cc:347] Connected to a master server at 127.13.38.62:33717
I20260812 17:52:31.889899 11839 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:31.890566 11839 heartbeater.cc:511] Master 127.13.38.62:33717 requested a full tablet report, sending...
I20260812 17:52:31.892040 11360 ts_manager.cc:194] Registered new tserver with Master: 2f421b8849b74ba39e86c3b346b64fc5 (127.13.38.2:40357)
I20260812 17:52:31.892676 11360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.2@KRBTEST.COM'} at 127.13.38.2:47939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
Entry for principal kudu/127.13.38.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab.
I20260812 17:52:31.907433 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:33717
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:32.037765 11848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:32.038072 11848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:32.038121 11848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:32.042886 11848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:32.042986 11848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:32.043071 11848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:52:32.049219 11848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.3.keytab
--principal=kudu/127.13.38.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:33717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:32.050449 11848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:32.051610 11848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:32.062884 11854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:32.062928 11853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:32.062929 11848 server_base.cc:1034] running on GCE node
W20260812 17:52:32.062884 11857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:32.063764 11848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:32.064483 11848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:32.065632 11848 hybrid_clock.cc:648] HybridClock initialized: now 1786557152065606 us; error 42 us; skew 500 ppm
Aug 12 17:52:32 dist-test-slave-sb2z krb5kdc[11301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557152, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:32.068866 11848 init.cc:377] Logged in from keytab as kudu/127.13.38.3@KRBTEST.COM (short username kudu)
I20260812 17:52:32.069905 11848 webserver.cc:460] Webserver started at http://127.13.38.3:33549/ using document root <none> and password file <none>
I20260812 17:52:32.070524 11848 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:32.070595 11848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:32.070781 11848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:32.073125 11848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "63bddf6a4ce340e7aecfbe256bf13390"
format_stamp: "Formatted at 2026-08-12 17:52:32 on dist-test-slave-sb2z"
server_key: "4a75769e831150ecfb3f10e76f849958"
server_key_iv: "d0266e2d88ba1ae6a266929058aee46b"
server_key_version: "encryptionkey@0"
I20260812 17:52:32.073849 11848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "63bddf6a4ce340e7aecfbe256bf13390"
format_stamp: "Formatted at 2026-08-12 17:52:32 on dist-test-slave-sb2z"
server_key: "4a75769e831150ecfb3f10e76f849958"
server_key_iv: "d0266e2d88ba1ae6a266929058aee46b"
server_key_version: "encryptionkey@0"
I20260812 17:52:32.079218 11848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 17:52:32.082727 11866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:32.084235 11848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:32.084355 11848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "63bddf6a4ce340e7aecfbe256bf13390"
format_stamp: "Formatted at 2026-08-12 17:52:32 on dist-test-slave-sb2z"
server_key: "4a75769e831150ecfb3f10e76f849958"
server_key_iv: "d0266e2d88ba1ae6a266929058aee46b"
server_key_version: "encryptionkey@0"
I20260812 17:52:32.084468 11848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:32.106369 11848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:32.109867 11848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:32.110046 11848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:32.110657 11848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:32.111588 11848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:32.111656 11848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:32.111701 11848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:32.111720 11848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:32.121452 11848 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:44579
I20260812 17:52:32.121488 12033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:44579 every 8 connection(s)
I20260812 17:52:32.122481 11848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:52:32.126940 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 11848
I20260812 17:52:32.127108 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:52:32.127452 13464 external_mini_cluster.cc:1428] Setting key 605f5cb4a93b7ac6d1153acd45aeb372
Aug 12 17:52:32 dist-test-slave-sb2z krb5kdc[11301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557152, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.3@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:52:32.139242 11421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.124699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:55755 (local address 127.13.38.62:33717)
0812 17:52:32.124892 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:32.124896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:32.125923 (+ 1027us) server_negotiation.cc:407] Connection header received
0812 17:52:32.127767 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:32.127770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:32.127843 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:32.127955 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:32.129604 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.130301 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.131808 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.132038 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.132138 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.135205 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:32.135209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:32.135211 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:32.135242 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:32.136996 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:32.137670 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:32.137673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:32.137674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:32.137760 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:32.138117 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:32.138119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:32.138120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:32.138315 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:32.138396 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:32.138977 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 17:52:32.139089 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 17:52:32.140230 12036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.125204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:33717 (local address 127.13.38.3:55755)
0812 17:52:32.125739 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:32.125789 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:32.127441 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:32.128156 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:32.128164 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:32.128715 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:32.129401 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:32.129423 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.130483 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.130487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:32.131641 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:32.131656 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.132229 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.132232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:32.132407 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.133109 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:52:32.133133 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:32.134914 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:32.137169 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:32.137177 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:32.137193 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:32.137563 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:32.137849 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:52:32.137851 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:52:32.137852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:52:32.138011 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:52:32.138446 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:32.138452 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:32.138747 (+ 295us) client_negotiation.cc:769] Sending connection context
0812 17:52:32.138933 (+ 186us) client_negotiation.cc:240] Negotiation successful
0812 17:52:32.139249 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":133,"threads_started":1}
I20260812 17:52:32.141803 12034 heartbeater.cc:347] Connected to a master server at 127.13.38.62:33717
I20260812 17:52:32.142091 12034 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:32.142679 12034 heartbeater.cc:511] Master 127.13.38.62:33717 requested a full tablet report, sending...
I20260812 17:52:32.143841 11359 ts_manager.cc:194] Registered new tserver with Master: 63bddf6a4ce340e7aecfbe256bf13390 (127.13.38.3:44579)
I20260812 17:52:32.144516 11359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.38.3@KRBTEST.COM'} at 127.13.38.3:55755
I20260812 17:52:32.154556 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:52:32.165490 11421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.156848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43984 (local address 127.13.38.62:33717)
0812 17:52:32.157088 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:32.157093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:32.157212 (+ 119us) server_negotiation.cc:407] Connection header received
0812 17:52:32.157293 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:32.157296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:32.157350 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:32.157442 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:32.158309 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.158864 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.159926 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.160163 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.160262 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.161827 (+ 1565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:32.161830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:32.161832 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:32.161862 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:32.163501 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:32.164063 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:32.164066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:32.164067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:32.164163 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:32.164460 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:32.164462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:32.164464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:32.164710 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:32.164822 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:32.165067 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 17:52:32.165286 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260812 17:52:32.168982 11359 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:52:32.172457 11359 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:52:32.190479 12049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.183060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:44579 (local address 127.0.0.1:59816)
0812 17:52:32.184206 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:32.184263 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:32.184457 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:32.185322 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:32.185325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:32.185344 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:32.185467 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:32.185485 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.187157 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.187160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:32.188510 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:32.188516 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.189881 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.189885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:32.190092 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.190144 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 17:52:32.190215 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 17:52:32.190316 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"mutex_wait_us":162,"thread_start_us":116,"threads_started":1}
I20260812 17:52:32.190601 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.184415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35006 (local address 127.13.38.1:33563)
0812 17:52:32.184834 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:32.184841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:32.184864 (+ 23us) server_negotiation.cc:407] Connection header received
0812 17:52:32.184944 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:32.184953 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:32.185164 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:32.185322 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:32.185699 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.186479 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.188656 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.189530 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.189640 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.189808 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:32.190317 (+ 509us) server_negotiation.cc:299] Negotiation successful
0812 17:52:32.190425 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":97,"threads_started":1}
I20260812 17:52:32.190971 12050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.183130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59816 (local address 127.13.38.3:44579)
0812 17:52:32.183925 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:32.183941 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:32.184368 (+ 427us) server_negotiation.cc:407] Connection header received
0812 17:52:32.184657 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:32.184664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:32.184935 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:32.185087 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:32.185630 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.186977 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.188654 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.189522 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.189640 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.189808 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:32.190733 (+ 925us) server_negotiation.cc:299] Negotiation successful
0812 17:52:32.190846 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":167,"threads_started":1}
I20260812 17:52:32.190480 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.183242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:40357 (local address 127.0.0.1:37484)
0812 17:52:32.184627 (+ 1385us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:32.184643 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:32.184726 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:32.185240 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:32.185246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:32.185270 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:32.185440 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:32.185449 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.187168 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.187172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:32.188429 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:32.188441 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.189661 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.189665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:32.189867 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.189910 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 17:52:32.190084 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 17:52:32.190240 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1321,"mutex_wait_us":44,"thread_start_us":107,"threads_started":1}
I20260812 17:52:32.191702 12054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.183981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.13.38.2:40357)
0812 17:52:32.184827 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:32.184845 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:32.184865 (+ 20us) server_negotiation.cc:407] Connection header received
0812 17:52:32.184930 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:32.184935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:32.185073 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:32.185247 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:32.185568 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.186984 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.188638 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.189513 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.189775 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.189949 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:32.190121 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 17:52:32.191453 (+ 1332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":135,"threads_started":1}
I20260812 17:52:32.191811 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.183007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:33563 (local address 127.0.0.1:35006)
0812 17:52:32.184205 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:32.184263 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:32.184401 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:32.185454 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:32.185457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:32.185477 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:32.185570 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:32.185581 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.186614 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.186619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:32.188492 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:32.188505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.189881 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.189885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:32.190096 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.190137 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 17:52:32.190215 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 17:52:32.191683 (+ 1468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"mutex_wait_us":1496,"spinlock_wait_cycles":21376,"thread_start_us":113,"threads_started":1}
I20260812 17:52:32.194875 11934 tablet_service.cc:1467] Processing CreateTablet for tablet 2629bdcdcf264022b57afa5ae8446697 (DEFAULT_TABLE table=test-table [id=a620122bfa8146fc84fb216003b4295d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:32.194877 11741 tablet_service.cc:1467] Processing CreateTablet for tablet 2629bdcdcf264022b57afa5ae8446697 (DEFAULT_TABLE table=test-table [id=a620122bfa8146fc84fb216003b4295d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:32.196161 11741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2629bdcdcf264022b57afa5ae8446697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:32.196206 11934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2629bdcdcf264022b57afa5ae8446697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:32.197218 11546 tablet_service.cc:1467] Processing CreateTablet for tablet 2629bdcdcf264022b57afa5ae8446697 (DEFAULT_TABLE table=test-table [id=a620122bfa8146fc84fb216003b4295d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:32.198518 11546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2629bdcdcf264022b57afa5ae8446697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:32.203392 12056 tablet_bootstrap.cc:492] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5: Bootstrap starting.
I20260812 17:52:32.205286 12058 tablet_bootstrap.cc:492] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390: Bootstrap starting.
I20260812 17:52:32.206336 12056 tablet_bootstrap.cc:654] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:32.207643 12056 log.cc:824] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:32.207669 12059 tablet_bootstrap.cc:492] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f: Bootstrap starting.
I20260812 17:52:32.207757 12058 tablet_bootstrap.cc:654] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:32.209111 12058 log.cc:824] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:32.209830 12056 tablet_bootstrap.cc:492] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5: No bootstrap required, opened a new log
I20260812 17:52:32.210081 12056 ts_tablet_manager.cc:1397] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 17:52:32.210119 12059 tablet_bootstrap.cc:654] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:32.210974 12059 log.cc:824] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:32.211481 12058 tablet_bootstrap.cc:492] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390: No bootstrap required, opened a new log
I20260812 17:52:32.211740 12058 ts_tablet_manager.cc:1397] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260812 17:52:32.213251 12059 tablet_bootstrap.cc:492] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f: No bootstrap required, opened a new log
I20260812 17:52:32.213508 12059 ts_tablet_manager.cc:1397] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 17:52:32.213624 12056 raft_consensus.cc:348] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63bddf6a4ce340e7aecfbe256bf13390" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44579 } } peers { permanent_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 33563 } } peers { permanent_uuid: "2f421b8849b74ba39e86c3b346b64fc5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 40357 } }
I20260812 17:52:32.214321 12056 raft_consensus.cc:374] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:32.214391 12056 raft_consensus.cc:729] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f421b8849b74ba39e86c3b346b64fc5, State: Initialized, Role: FOLLOWER
I20260812 17:52:32.214880 12056 consensus_queue.cc:260] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63bddf6a4ce340e7aecfbe256bf13390" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44579 } } peers { permanent_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 33563 } } peers { permanent_uuid: "2f421b8849b74ba39e86c3b346b64fc5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 40357 } }
I20260812 17:52:32.215711 11839 heartbeater.cc:503] Master 127.13.38.62:33717 was elected leader, sending a full tablet report...
I20260812 17:52:32.216094 12056 ts_tablet_manager.cc:1428] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 17:52:32.215965 12058 raft_consensus.cc:348] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63bddf6a4ce340e7aecfbe256bf13390" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44579 } } peers { permanent_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 33563 } } peers { permanent_uuid: "2f421b8849b74ba39e86c3b346b64fc5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 40357 } }
I20260812 17:52:32.216387 12058 raft_consensus.cc:374] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:32.216459 12058 raft_consensus.cc:729] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63bddf6a4ce340e7aecfbe256bf13390, State: Initialized, Role: FOLLOWER
I20260812 17:52:32.217000 12058 consensus_queue.cc:260] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63bddf6a4ce340e7aecfbe256bf13390" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44579 } } peers { permanent_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 33563 } } peers { permanent_uuid: "2f421b8849b74ba39e86c3b346b64fc5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 40357 } }
I20260812 17:52:32.217756 12034 heartbeater.cc:503] Master 127.13.38.62:33717 was elected leader, sending a full tablet report...
I20260812 17:52:32.218083 12058 ts_tablet_manager.cc:1428] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260812 17:52:32.220432 12059 raft_consensus.cc:348] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63bddf6a4ce340e7aecfbe256bf13390" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44579 } } peers { permanent_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 33563 } } peers { permanent_uuid: "2f421b8849b74ba39e86c3b346b64fc5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 40357 } }
I20260812 17:52:32.220777 12059 raft_consensus.cc:374] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:32.220856 12059 raft_consensus.cc:729] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51e6878fe3ed44c39ec157f36a8d1c9f, State: Initialized, Role: FOLLOWER
I20260812 17:52:32.221411 12059 consensus_queue.cc:260] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63bddf6a4ce340e7aecfbe256bf13390" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44579 } } peers { permanent_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 33563 } } peers { permanent_uuid: "2f421b8849b74ba39e86c3b346b64fc5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 40357 } }
I20260812 17:52:32.222268 11641 heartbeater.cc:503] Master 127.13.38.62:33717 was elected leader, sending a full tablet report...
I20260812 17:52:32.222508 12059 ts_tablet_manager.cc:1428] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260812 17:52:32.351787 12064 raft_consensus.cc:482] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:52:32.352087 12064 raft_consensus.cc:504] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63bddf6a4ce340e7aecfbe256bf13390" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44579 } } peers { permanent_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 33563 } } peers { permanent_uuid: "2f421b8849b74ba39e86c3b346b64fc5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 40357 } }
I20260812 17:52:32.353556 12064 leader_election.cc:288] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63bddf6a4ce340e7aecfbe256bf13390 (127.13.38.3:44579), 51e6878fe3ed44c39ec157f36a8d1c9f (127.13.38.1:33563)
I20260812 17:52:32.359112 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.354229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:49157 (local address 127.13.38.1:33563)
0812 17:52:32.354403 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:32.354408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:32.354538 (+ 130us) server_negotiation.cc:407] Connection header received
0812 17:52:32.354633 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:32.354636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:32.354679 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:32.354776 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:32.355026 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.355753 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.357526 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.358248 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.358348 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.358407 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:32.358882 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 17:52:32.358976 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 17:52:32.359112 12050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.354020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:58701 (local address 127.13.38.3:44579)
0812 17:52:32.354162 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:32.354167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:32.354186 (+ 19us) server_negotiation.cc:407] Connection header received
0812 17:52:32.354250 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:32.354253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:32.354320 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:32.354470 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:32.354787 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.355643 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.357526 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.358176 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.358314 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.358419 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:32.358880 (+ 461us) server_negotiation.cc:299] Negotiation successful
0812 17:52:32.358970 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 17:52:32.359122 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.353850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:44579 (local address 127.13.38.2:58701)
0812 17:52:32.353992 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:32.354009 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:32.354129 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:32.354480 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:32.354486 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:32.354523 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:32.354669 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:32.354675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.355771 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.355774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:32.357360 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:32.357373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.358340 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.358344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:32.358647 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.358719 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 17:52:32.358806 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 17:52:32.358904 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":206}
I20260812 17:52:32.359130 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.353861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:33563 (local address 127.13.38.2:49157)
0812 17:52:32.354419 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:32.354435 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:32.354538 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:32.354791 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:32.354793 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:32.354812 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:32.354909 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:32.354913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.355856 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.355859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:32.357360 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:32.357373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.358354 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.358357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:32.358652 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.358723 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 17:52:32.358806 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 17:52:32.358904 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":315,"thread_start_us":119,"threads_started":1}
I20260812 17:52:32.359886 11571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2629bdcdcf264022b57afa5ae8446697" candidate_uuid: "2f421b8849b74ba39e86c3b346b64fc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f" is_pre_election: true
I20260812 17:52:32.360245 11967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2629bdcdcf264022b57afa5ae8446697" candidate_uuid: "2f421b8849b74ba39e86c3b346b64fc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63bddf6a4ce340e7aecfbe256bf13390" is_pre_election: true
I20260812 17:52:32.360283 11571 raft_consensus.cc:2454] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f421b8849b74ba39e86c3b346b64fc5 in term 0.
I20260812 17:52:32.360514 11967 raft_consensus.cc:2454] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f421b8849b74ba39e86c3b346b64fc5 in term 0.
I20260812 17:52:32.360858 11673 leader_election.cc:302] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f421b8849b74ba39e86c3b346b64fc5, 51e6878fe3ed44c39ec157f36a8d1c9f; no voters:
I20260812 17:52:32.361240 12064 raft_consensus.cc:2781] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:52:32.361336 12064 raft_consensus.cc:482] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:52:32.361368 12064 raft_consensus.cc:3037] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:32.362505 12064 raft_consensus.cc:504] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63bddf6a4ce340e7aecfbe256bf13390" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44579 } } peers { permanent_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 33563 } } peers { permanent_uuid: "2f421b8849b74ba39e86c3b346b64fc5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 40357 } }
I20260812 17:52:32.362839 12064 leader_election.cc:288] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [CANDIDATE]: Term 1 election: Requested vote from peers 63bddf6a4ce340e7aecfbe256bf13390 (127.13.38.3:44579), 51e6878fe3ed44c39ec157f36a8d1c9f (127.13.38.1:33563)
I20260812 17:52:32.363319 11967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2629bdcdcf264022b57afa5ae8446697" candidate_uuid: "2f421b8849b74ba39e86c3b346b64fc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63bddf6a4ce340e7aecfbe256bf13390"
I20260812 17:52:32.363325 11571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2629bdcdcf264022b57afa5ae8446697" candidate_uuid: "2f421b8849b74ba39e86c3b346b64fc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f"
I20260812 17:52:32.363502 11967 raft_consensus.cc:3037] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:32.363502 11571 raft_consensus.cc:3037] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:32.364945 11967 raft_consensus.cc:2454] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f421b8849b74ba39e86c3b346b64fc5 in term 1.
I20260812 17:52:32.364948 11571 raft_consensus.cc:2454] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f421b8849b74ba39e86c3b346b64fc5 in term 1.
I20260812 17:52:32.365361 11673 leader_election.cc:302] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f421b8849b74ba39e86c3b346b64fc5, 51e6878fe3ed44c39ec157f36a8d1c9f; no voters:
I20260812 17:52:32.365566 12064 raft_consensus.cc:2781] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:32.365857 12064 raft_consensus.cc:686] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 1 LEADER]: Becoming Leader. State: Replica: 2f421b8849b74ba39e86c3b346b64fc5, State: Running, Role: LEADER
I20260812 17:52:32.366282 12064 consensus_queue.cc:237] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63bddf6a4ce340e7aecfbe256bf13390" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44579 } } peers { permanent_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 33563 } } peers { permanent_uuid: "2f421b8849b74ba39e86c3b346b64fc5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 40357 } }
I20260812 17:52:32.369652 11360 catalog_manager.cc:5495] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f421b8849b74ba39e86c3b346b64fc5 (127.13.38.2). New cstate: current_term: 1 leader_uuid: "2f421b8849b74ba39e86c3b346b64fc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63bddf6a4ce340e7aecfbe256bf13390" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 33563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f421b8849b74ba39e86c3b346b64fc5" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 40357 } health_report { overall_health: HEALTHY } } }
W20260812 17:52:32.372748 11840 tablet.cc:2369] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:52:32.372849 11642 tablet.cc:2369] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 17:52:32.373634 12035 tablet.cc:2369] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:52:32.384214 12054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.380321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37492 (local address 127.13.38.2:40357)
0812 17:52:32.380511 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:32.380515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:32.380531 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:52:32.380620 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:32.380622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:32.380680 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:32.380781 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:52:32.381138 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.381670 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.382575 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.382798 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.382883 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.383279 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:52:32.383818 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:52:32.383918 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:32.384009 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 17:52:32.384070 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:52:32.391148 11571 raft_consensus.cc:1261] T 2629bdcdcf264022b57afa5ae8446697 P 51e6878fe3ed44c39ec157f36a8d1c9f [term 1 FOLLOWER]: Refusing update from remote peer 2f421b8849b74ba39e86c3b346b64fc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:52:32.391148 11967 raft_consensus.cc:1261] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390 [term 1 FOLLOWER]: Refusing update from remote peer 2f421b8849b74ba39e86c3b346b64fc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:52:32.392014 12064 consensus_queue.cc:1035] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63bddf6a4ce340e7aecfbe256bf13390" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 44579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:52:32.392256 12068 consensus_queue.cc:1035] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e6878fe3ed44c39ec157f36a8d1c9f" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 33563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:52:32.401974 12074 mvcc.cc:205] Tried to move back new op lower bound from 7317738096184889344 to 7317738096095248384. Current Snapshot: MvccSnapshot[applied={T|T < 7317738096184889344}]
I20260812 17:52:32.409823 11360 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:52:32.410048 11360 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:52:32.412897 11360 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3dd8ea01129457eacf516e66ab70a77: Sending DeleteTablet for 3 replicas of tablet 2629bdcdcf264022b57afa5ae8446697
I20260812 17:52:32.413932 11546 tablet_service.cc:1514] Processing DeleteTablet for tablet 2629bdcdcf264022b57afa5ae8446697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:32 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:35006
I20260812 17:52:32.414006 11934 tablet_service.cc:1514] Processing DeleteTablet for tablet 2629bdcdcf264022b57afa5ae8446697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:32 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:59816
I20260812 17:52:32.414012 11741 tablet_service.cc:1514] Processing DeleteTablet for tablet 2629bdcdcf264022b57afa5ae8446697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:32 UTC) from {username='kudu', principal='kudu/127.13.38.62@KRBTEST.COM'} at 127.0.0.1:37484
I20260812 17:52:32.414670 12088 tablet_replica.cc:330] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390: stopping tablet replica
I20260812 17:52:32.414701 12089 tablet_replica.cc:330] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5: stopping tablet replica
I20260812 17:52:32.415053 12089 raft_consensus.cc:2229] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 17:52:32.415055 12088 raft_consensus.cc:2229] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:52:32.415397 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 11451
I20260812 17:52:32.415418 12088 raft_consensus.cc:2258] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:52:32.422606 12088 ts_tablet_manager.cc:1905] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:52:32.423897 12089 raft_consensus.cc:2258] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:52:32.426694 12089 ts_tablet_manager.cc:1905] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:52:32.427189 12088 ts_tablet_manager.cc:1918] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:52:32.427323 12088 log.cc:1196] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/2629bdcdcf264022b57afa5ae8446697
I20260812 17:52:32.427733 12088 ts_tablet_manager.cc:1939] T 2629bdcdcf264022b57afa5ae8446697 P 63bddf6a4ce340e7aecfbe256bf13390: Deleting consensus metadata
I20260812 17:52:32.429315 11340 catalog_manager.cc:4841] TS 63bddf6a4ce340e7aecfbe256bf13390 (127.13.38.3:44579): tablet 2629bdcdcf264022b57afa5ae8446697 (table test-table [id=a620122bfa8146fc84fb216003b4295d]) successfully deleted
I20260812 17:52:32.429705 12089 ts_tablet_manager.cc:1918] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:52:32.429831 12089 log.cc:1196] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/2629bdcdcf264022b57afa5ae8446697
I20260812 17:52:32.430169 12089 ts_tablet_manager.cc:1939] T 2629bdcdcf264022b57afa5ae8446697 P 2f421b8849b74ba39e86c3b346b64fc5: Deleting consensus metadata
I20260812 17:52:32.432264 11341 catalog_manager.cc:4841] TS 2f421b8849b74ba39e86c3b346b64fc5 (127.13.38.2:40357): tablet 2629bdcdcf264022b57afa5ae8446697 (table test-table [id=a620122bfa8146fc84fb216003b4295d]) successfully deleted
W20260812 17:52:32.434412 11339 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:33563 (error 108)
I20260812 17:52:32.434660 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 11651
I20260812 17:52:32.438033 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.437650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:33563 (local address 127.0.0.1:35018)
0812 17:52:32.437851 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:32.437927 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:33563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260812 17:52:32.438241 11339 catalog_manager.cc:4584] TS 51e6878fe3ed44c39ec157f36a8d1c9f (127.13.38.1:33563): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2629bdcdcf264022b57afa5ae8446697: Network error: Client connection negotiation failed: client connection to 127.13.38.1:33563: connect: Connection refused (error 111)
I20260812 17:52:32.447224 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 11848
I20260812 17:52:32.463750 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 11316
2026-08-12T17:52:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2843 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T17:52:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:32Z Disabled control of system clock
I20260812 17:52:32.507858 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:38187
--webserver_interface=127.13.38.62
--webserver_port=0
--builtin_ntp_servers=127.13.38.20:39165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:38187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 17:52:32.630354 12098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:32.630685 12098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:32.630725 12098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:32.635010 12098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:32.635102 12098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:32.635119 12098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:32.635138 12098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:32.635154 12098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:32.640900 12098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:38187
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:38187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:32.642109 12098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:32.643603 12098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:32.653228 12104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:32.653208 12103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:32.653208 12107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:32.653414 12098 server_base.cc:1034] running on GCE node
I20260812 17:52:32.654337 12098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:32.655856 12098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:32.657099 12098 hybrid_clock.cc:648] HybridClock initialized: now 1786557152657061 us; error 53 us; skew 500 ppm
I20260812 17:52:32.663081 12098 webserver.cc:460] Webserver started at http://127.13.38.62:37533/ using document root <none> and password file <none>
I20260812 17:52:32.664088 12098 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:32.664186 12098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:32.664479 12098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:32.666890 12098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "ad238463d48f4e35abf380e5447d3808"
format_stamp: "Formatted at 2026-08-12 17:52:32 on dist-test-slave-sb2z"
server_key: "148802e38bf01c497ffaddb019c1d3b0"
server_key_iv: "2889b2256efbd903830c47dc8c2b3d1b"
server_key_version: "encryptionkey@0"
I20260812 17:52:32.667665 12098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "ad238463d48f4e35abf380e5447d3808"
format_stamp: "Formatted at 2026-08-12 17:52:32 on dist-test-slave-sb2z"
server_key: "148802e38bf01c497ffaddb019c1d3b0"
server_key_iv: "2889b2256efbd903830c47dc8c2b3d1b"
server_key_version: "encryptionkey@0"
I20260812 17:52:32.672225 12098 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 17:52:32.675379 12116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:32.677027 12098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 17:52:32.677224 12098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "ad238463d48f4e35abf380e5447d3808"
format_stamp: "Formatted at 2026-08-12 17:52:32 on dist-test-slave-sb2z"
server_key: "148802e38bf01c497ffaddb019c1d3b0"
server_key_iv: "2889b2256efbd903830c47dc8c2b3d1b"
server_key_version: "encryptionkey@0"
I20260812 17:52:32.677374 12098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:32.694052 12098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:32.694931 12098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:32.695180 12098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:32.703483 12191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:38187 every 8 connection(s)
I20260812 17:52:32.703469 12098 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:38187
I20260812 17:52:32.704833 12098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:32.705636 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 12098
I20260812 17:52:32.705780 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:32.706138 13464 external_mini_cluster.cc:1428] Setting key 3ea228c9a1da366355d0f79a33ebf99a
I20260812 17:52:32.708811 12192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:32.715739 12192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808: Bootstrap starting.
I20260812 17:52:32.716421 12195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:32.707803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59830 (local address 127.13.38.62:38187)
0812 17:52:32.708480 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:32.708497 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:32.708535 (+ 38us) server_negotiation.cc:407] Connection header received
0812 17:52:32.709290 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:32.709311 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:32.709743 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:32.710052 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:32.710415 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.711387 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.712660 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:32.713147 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:32.713280 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:32.714258 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:32.714263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:32.714278 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:32.714700 (+ 422us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:32.714764 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:32.714786 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:32.714916 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:32.715043 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 17:52:32.715377 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":153,"threads_started":1}
I20260812 17:52:32.718357 12192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:32.719199 12192 log.cc:824] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:32.721643 12192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808: No bootstrap required, opened a new log
I20260812 17:52:32.725436 12192 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad238463d48f4e35abf380e5447d3808" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38187 } }
I20260812 17:52:32.725704 12192 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:32.725769 12192 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad238463d48f4e35abf380e5447d3808, State: Initialized, Role: FOLLOWER
I20260812 17:52:32.726233 12192 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad238463d48f4e35abf380e5447d3808" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38187 } }
I20260812 17:52:32.726359 12192 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:32.726444 12192 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:32.726544 12192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:32.727645 12192 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad238463d48f4e35abf380e5447d3808" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38187 } }
I20260812 17:52:32.728016 12192 leader_election.cc:302] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad238463d48f4e35abf380e5447d3808; no voters:
I20260812 17:52:32.728384 12192 leader_election.cc:288] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:32.728636 12197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:32.728919 12197 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [term 1 LEADER]: Becoming Leader. State: Replica: ad238463d48f4e35abf380e5447d3808, State: Running, Role: LEADER
I20260812 17:52:32.729324 12197 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad238463d48f4e35abf380e5447d3808" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38187 } }
I20260812 17:52:32.729744 12192 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:32.731244 12198 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad238463d48f4e35abf380e5447d3808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad238463d48f4e35abf380e5447d3808" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38187 } } }
I20260812 17:52:32.731304 12199 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad238463d48f4e35abf380e5447d3808. Latest consensus state: current_term: 1 leader_uuid: "ad238463d48f4e35abf380e5447d3808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad238463d48f4e35abf380e5447d3808" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 38187 } } }
I20260812 17:52:32.731392 12198 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:32.731477 12199 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808 [sys.catalog]: This master's current role is: LEADER
W20260812 17:52:32.734934 12214 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:52:32.735023 12214 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:52:32.735126 12210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:32.735972 12210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:32.742165 12210 catalog_manager.cc:1347] Generated new cluster ID: 542022bdd6ad424c8ea795f0358ba131
I20260812 17:52:32.742262 12210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:32.754462 12210 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:32.755916 12210 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:32.771296 12210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808: Generated new TSK 0
I20260812 17:52:32.772173 12210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:32.784965 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.1:0
--local_ip_for_outbound_sockets=127.13.38.1
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:38187
--builtin_ntp_servers=127.13.38.20:39165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 17:52:32.912988 12218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:32.913304 12218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:32.913347 12218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:32.917558 12218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:32.917652 12218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:32.917716 12218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.1
I20260812 17:52:32.923019 12218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.38.1
--webserver_port=0
--tserver_master_addrs=127.13.38.62:38187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.1
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:32.924299 12218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:32.925714 12218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:32.938238 12227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:32.938239 12234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:32.938326 12229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:32.938314 12218 server_base.cc:1034] running on GCE node
I20260812 17:52:32.939057 12218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:32.939862 12218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:32.941118 12218 hybrid_clock.cc:648] HybridClock initialized: now 1786557152941060 us; error 78 us; skew 500 ppm
I20260812 17:52:32.943409 12218 webserver.cc:460] Webserver started at http://127.13.38.1:34485/ using document root <none> and password file <none>
I20260812 17:52:32.944197 12218 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:32.944275 12218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:32.944530 12218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:32.946996 12218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/data/instance:
uuid: "d2db2f8d625c4d109c8fa4dd17981fc8"
format_stamp: "Formatted at 2026-08-12 17:52:32 on dist-test-slave-sb2z"
server_key: "cbb4bea95c48cfd542c63da0d3ccac13"
server_key_iv: "861a9acf1974acc5d7bacb42cd8165b3"
server_key_version: "encryptionkey@0"
I20260812 17:52:32.947757 12218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance:
uuid: "d2db2f8d625c4d109c8fa4dd17981fc8"
format_stamp: "Formatted at 2026-08-12 17:52:32 on dist-test-slave-sb2z"
server_key: "cbb4bea95c48cfd542c63da0d3ccac13"
server_key_iv: "861a9acf1974acc5d7bacb42cd8165b3"
server_key_version: "encryptionkey@0"
I20260812 17:52:32.952389 12218 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:52:32.956221 12242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:32.957862 12218 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:52:32.958007 12218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/wal
uuid: "d2db2f8d625c4d109c8fa4dd17981fc8"
format_stamp: "Formatted at 2026-08-12 17:52:32 on dist-test-slave-sb2z"
server_key: "cbb4bea95c48cfd542c63da0d3ccac13"
server_key_iv: "861a9acf1974acc5d7bacb42cd8165b3"
server_key_version: "encryptionkey@0"
I20260812 17:52:32.958194 12218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:32.985173 12218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:32.986083 12218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:32.986333 12218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:32.987063 12218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:32.988288 12218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:32.988346 12218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:32.988423 12218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:32.988461 12218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:32.999650 12218 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.1:42581
I20260812 17:52:32.999708 12405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.1:42581 every 8 connection(s)
I20260812 17:52:33.000913 12218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/data/info.pb
I20260812 17:52:33.004345 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 12218
I20260812 17:52:33.004503 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-0/wal/instance
I20260812 17:52:33.004855 13464 external_mini_cluster.cc:1428] Setting key e19e94837662e5ff68ec178af9e68639
I20260812 17:52:33.007423 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.2:0
--local_ip_for_outbound_sockets=127.13.38.2
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:38187
--builtin_ntp_servers=127.13.38.20:39165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 17:52:33.013196 12195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.003101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:52617 (local address 127.13.38.62:38187)
0812 17:52:33.003278 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:33.003282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:33.004931 (+ 1649us) server_negotiation.cc:407] Connection header received
0812 17:52:33.006960 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:33.006963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:33.007017 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:33.007134 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:33.008588 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.009251 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.010682 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.010946 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.011090 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.012517 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:33.012522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:33.012525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:33.012551 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:33.012574 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:33.012583 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:33.012672 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:33.012893 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 17:52:33.013016 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 17:52:33.014213 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.003644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38187 (local address 127.13.38.1:52617)
0812 17:52:33.004687 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:33.004754 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:33.006687 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:33.007385 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:33.007397 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:33.007558 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:33.008387 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.008413 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.009428 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.009433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.010518 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.010531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.011151 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.011155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.011356 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.012175 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:52:33.012177 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:33.012229 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:52:33.012313 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:33.012690 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:33.012703 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:33.012718 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 17:52:33.012869 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 17:52:33.013233 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":149,"threads_started":1}
I20260812 17:52:33.015983 12406 heartbeater.cc:347] Connected to a master server at 127.13.38.62:38187
I20260812 17:52:33.016299 12406 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:33.016953 12406 heartbeater.cc:511] Master 127.13.38.62:38187 requested a full tablet report, sending...
I20260812 17:52:33.018821 12144 ts_manager.cc:194] Registered new tserver with Master: d2db2f8d625c4d109c8fa4dd17981fc8 (127.13.38.1:42581)
I20260812 17:52:33.020256 12144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.38.1:52617
W20260812 17:52:33.142859 12411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:33.143237 12411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:33.143317 12411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:33.147821 12411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:33.147935 12411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:33.148041 12411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.2
I20260812 17:52:33.153286 12411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.38.2
--webserver_port=0
--tserver_master_addrs=127.13.38.62:38187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.2
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:33.154639 12411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:33.156656 12411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:33.168045 12419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:33.168097 12417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:33.168045 12422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:33.168308 12411 server_base.cc:1034] running on GCE node
I20260812 17:52:33.168850 12411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:33.169663 12411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:33.170903 12411 hybrid_clock.cc:648] HybridClock initialized: now 1786557153170908 us; error 50 us; skew 500 ppm
I20260812 17:52:33.173413 12411 webserver.cc:460] Webserver started at http://127.13.38.2:45561/ using document root <none> and password file <none>
I20260812 17:52:33.174139 12411 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:33.174259 12411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:33.174538 12411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:33.177253 12411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/data/instance:
uuid: "037a8609ccfc4ce8b4173e26d425c84b"
format_stamp: "Formatted at 2026-08-12 17:52:33 on dist-test-slave-sb2z"
server_key: "e2e1a3a39ac571c6313d3a698dbc5ebe"
server_key_iv: "3d3fa1e23e2626ab3bcb9311ddb9b8a2"
server_key_version: "encryptionkey@0"
I20260812 17:52:33.177980 12411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance:
uuid: "037a8609ccfc4ce8b4173e26d425c84b"
format_stamp: "Formatted at 2026-08-12 17:52:33 on dist-test-slave-sb2z"
server_key: "e2e1a3a39ac571c6313d3a698dbc5ebe"
server_key_iv: "3d3fa1e23e2626ab3bcb9311ddb9b8a2"
server_key_version: "encryptionkey@0"
I20260812 17:52:33.182657 12411 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:52:33.186223 12430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:33.187840 12411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:33.188017 12411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/wal
uuid: "037a8609ccfc4ce8b4173e26d425c84b"
format_stamp: "Formatted at 2026-08-12 17:52:33 on dist-test-slave-sb2z"
server_key: "e2e1a3a39ac571c6313d3a698dbc5ebe"
server_key_iv: "3d3fa1e23e2626ab3bcb9311ddb9b8a2"
server_key_version: "encryptionkey@0"
I20260812 17:52:33.188205 12411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:33.198295 12411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:33.199151 12411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:33.199389 12411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:33.200156 12411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:33.201244 12411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:33.201324 12411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:33.201401 12411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:33.201452 12411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:33.212210 12411 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.2:46751
I20260812 17:52:33.212270 12602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.2:46751 every 8 connection(s)
I20260812 17:52:33.213316 12411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/data/info.pb
I20260812 17:52:33.216646 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 12411
I20260812 17:52:33.216778 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/wal/instance
I20260812 17:52:33.217212 13464 external_mini_cluster.cc:1428] Setting key c8cb8989b0ef5bec1b171043a7967494
I20260812 17:52:33.219810 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.38.3:0
--local_ip_for_outbound_sockets=127.13.38.3
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:38187
--builtin_ntp_servers=127.13.38.20:39165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 17:52:33.225502 12195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.215612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.2:46173 (local address 127.13.38.62:38187)
0812 17:52:33.215764 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:33.215768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:33.217315 (+ 1547us) server_negotiation.cc:407] Connection header received
0812 17:52:33.219109 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:33.219113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:33.219162 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:33.219257 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:33.220689 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.221244 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.222714 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.223139 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.223293 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.224674 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:33.224682 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:33.224689 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:33.224712 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:33.224743 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:33.224758 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:33.224855 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:33.225140 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 17:52:33.225244 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 17:52:33.226514 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.216042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38187 (local address 127.13.38.2:46173)
0812 17:52:33.217039 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:33.217110 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:33.218839 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:33.219508 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:33.219549 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:33.219697 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:33.220510 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.220535 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.221408 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.221415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.222553 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.222564 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.223327 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.223330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.223519 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.224313 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:52:33.224314 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:33.224362 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:52:33.224477 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:33.224912 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:33.224924 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:33.224932 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 17:52:33.225064 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 17:52:33.225345 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":173,"threads_started":1}
I20260812 17:52:33.227932 12603 heartbeater.cc:347] Connected to a master server at 127.13.38.62:38187
I20260812 17:52:33.228240 12603 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:33.228859 12603 heartbeater.cc:511] Master 127.13.38.62:38187 requested a full tablet report, sending...
I20260812 17:52:33.230190 12144 ts_manager.cc:194] Registered new tserver with Master: 037a8609ccfc4ce8b4173e26d425c84b (127.13.38.2:46751)
I20260812 17:52:33.230940 12144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.38.2:46173
W20260812 17:52:33.353689 12606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:33.354007 12606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:33.354049 12606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:33.358348 12606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:33.358425 12606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:33.358521 12606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.38.3
I20260812 17:52:33.364079 12606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.38.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.38.3
--webserver_port=0
--tserver_master_addrs=127.13.38.62:38187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.38.3
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:33.365206 12606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:33.366633 12606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:33.377321 12612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:33.377295 12615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:33.377300 12617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:33.377816 12606 server_base.cc:1034] running on GCE node
I20260812 17:52:33.378492 12606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:33.379375 12606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:33.380605 12606 hybrid_clock.cc:648] HybridClock initialized: now 1786557153380569 us; error 51 us; skew 500 ppm
I20260812 17:52:33.382786 12606 webserver.cc:460] Webserver started at http://127.13.38.3:37913/ using document root <none> and password file <none>
I20260812 17:52:33.383440 12606 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:33.383596 12606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:33.383862 12606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:33.386168 12606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/data/instance:
uuid: "6c2307e9c54348fb933552cfffcb8142"
format_stamp: "Formatted at 2026-08-12 17:52:33 on dist-test-slave-sb2z"
server_key: "5d84086ee2ca628572cab192c53901d2"
server_key_iv: "fa9381ef7cc1769ae45f97dedc410ad4"
server_key_version: "encryptionkey@0"
I20260812 17:52:33.386768 12606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance:
uuid: "6c2307e9c54348fb933552cfffcb8142"
format_stamp: "Formatted at 2026-08-12 17:52:33 on dist-test-slave-sb2z"
server_key: "5d84086ee2ca628572cab192c53901d2"
server_key_iv: "fa9381ef7cc1769ae45f97dedc410ad4"
server_key_version: "encryptionkey@0"
I20260812 17:52:33.390956 12606 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:52:33.394002 12626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:33.395624 12606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:33.395778 12606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/wal
uuid: "6c2307e9c54348fb933552cfffcb8142"
format_stamp: "Formatted at 2026-08-12 17:52:33 on dist-test-slave-sb2z"
server_key: "5d84086ee2ca628572cab192c53901d2"
server_key_iv: "fa9381ef7cc1769ae45f97dedc410ad4"
server_key_version: "encryptionkey@0"
I20260812 17:52:33.395915 12606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:33.425748 12606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:33.426636 12606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:33.426888 12606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:33.427634 12606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 17:52:33.428840 12606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 17:52:33.428901 12606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:33.428994 12606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 17:52:33.429039 12606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:33.440405 12606 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.3:42611
I20260812 17:52:33.440449 12789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.3:42611 every 8 connection(s)
I20260812 17:52:33.441476 12606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/data/info.pb
I20260812 17:52:33.450214 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 12606
I20260812 17:52:33.450387 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/wal/instance
I20260812 17:52:33.450760 13464 external_mini_cluster.cc:1428] Setting key 77ae2244c8e048af58e09bb8ef132bf8
I20260812 17:52:33.452778 12195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.443756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.3:48633 (local address 127.13.38.62:38187)
0812 17:52:33.443913 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:33.443917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:33.445090 (+ 1173us) server_negotiation.cc:407] Connection header received
0812 17:52:33.446621 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:33.446624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:33.446682 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:33.446772 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:33.448084 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.448642 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.449997 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.450403 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.450555 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.452133 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:33.452140 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:33.452146 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:33.452164 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:33.452199 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:33.452209 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:33.452277 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:33.452562 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 17:52:33.452647 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 17:52:33.453984 12796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.444201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:38187 (local address 127.13.38.3:48633)
0812 17:52:33.444904 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:33.444951 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:33.446342 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:33.446981 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:33.446989 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:33.447086 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:52:33.447891 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.447908 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.448780 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.448783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.449789 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.449800 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.450626 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.450631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.450856 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.451654 (+ 798us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 17:52:33.451656 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:52:33.451725 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 17:52:33.451838 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:52:33.452324 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:52:33.452336 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:52:33.452344 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 17:52:33.452502 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 17:52:33.452850 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":125,"threads_started":1}
I20260812 17:52:33.455454 12790 heartbeater.cc:347] Connected to a master server at 127.13.38.62:38187
I20260812 17:52:33.455793 12790 heartbeater.cc:464] Registering TS with master...
I20260812 17:52:33.456451 12790 heartbeater.cc:511] Master 127.13.38.62:38187 requested a full tablet report, sending...
I20260812 17:52:33.457777 12144 ts_manager.cc:194] Registered new tserver with Master: 6c2307e9c54348fb933552cfffcb8142 (127.13.38.3:42611)
I20260812 17:52:33.458750 12144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.38.3:48633
I20260812 17:52:33.466053 13464 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 17:52:33.472553 12195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.468344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.13.38.62:38187)
0812 17:52:33.468552 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:33.468557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:33.468728 (+ 171us) server_negotiation.cc:407] Connection header received
0812 17:52:33.468897 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:33.468900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:33.468994 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:33.469090 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:33.469408 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.469940 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.470823 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.471054 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.471172 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.472099 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:33.472103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:33.472106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:33.472120 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:33.472147 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:33.472155 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:33.472237 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:33.472332 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 17:52:33.472393 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 17:52:33.476132 12144 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 17:52:33.479344 12144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 17:52:33.495247 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.489364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58270 (local address 127.13.38.3:42611)
0812 17:52:33.489869 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:33.489884 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:33.490189 (+ 305us) server_negotiation.cc:407] Connection header received
0812 17:52:33.490296 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:33.490304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:33.490535 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:33.490722 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:33.491058 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.492260 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.493748 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.494495 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.494658 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.494902 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:33.494959 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 17:52:33.495072 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":169,"threads_started":1}
I20260812 17:52:33.495231 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.489169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:42611 (local address 127.0.0.1:58270)
0812 17:52:33.489822 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:33.489876 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:33.490041 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:33.490711 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:33.490718 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:33.490741 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:33.490898 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.490905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.492366 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.492372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.493604 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.493614 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.494631 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.494635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.494750 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.494789 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 17:52:33.494903 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 17:52:33.495045 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":86,"threads_started":1}
I20260812 17:52:33.496172 12808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.490112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:42581 (local address 127.0.0.1:56996)
0812 17:52:33.490548 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:33.490562 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:33.490653 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:33.491732 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:33.491735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:33.491755 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:33.491864 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.491871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.493423 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.493427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.494440 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.494450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.495731 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.495735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.495827 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.495858 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 17:52:33.495929 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 17:52:33.496001 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":18,"thread_start_us":98,"threads_started":1}
I20260812 17:52:33.496413 12816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.490660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56996 (local address 127.13.38.1:42581)
0812 17:52:33.491066 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:33.491073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:33.491099 (+ 26us) server_negotiation.cc:407] Connection header received
0812 17:52:33.491181 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:33.491187 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:33.491369 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:33.491763 (+ 394us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:33.491989 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.493274 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.494643 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.495590 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.495728 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.495928 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:33.495980 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 17:52:33.496104 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":119,"threads_started":1}
I20260812 17:52:33.498237 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.489370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:46751 (local address 127.0.0.1:40754)
0812 17:52:33.490374 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:33.490389 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:33.490467 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:33.491933 (+ 1466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:33.491936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:33.491957 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:33.492057 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.492063 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.493538 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.493541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.496652 (+ 3111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.496666 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.497820 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.497824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.497919 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.497944 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 17:52:33.498009 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 17:52:33.498090 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"mutex_wait_us":1,"thread_start_us":176,"threads_started":1}
I20260812 17:52:33.498312 12815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.490057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40754 (local address 127.13.38.2:46751)
0812 17:52:33.491186 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:33.491203 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:33.491223 (+ 20us) server_negotiation.cc:407] Connection header received
0812 17:52:33.491292 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:33.491296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:33.491448 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:33.491763 (+ 315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:33.492261 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.493420 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.496822 (+ 3402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.497679 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.497797 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.497989 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:33.498045 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 17:52:33.498141 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":190,"threads_started":1}
I20260812 17:52:33.499354 12310 tablet_service.cc:1467] Processing CreateTablet for tablet 643bdb5fe6d24d44ac2c501bfb72e6ba (DEFAULT_TABLE table=test-table [id=5666e36076a44fdfa8b11c74935da38b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:33.499241 12698 tablet_service.cc:1467] Processing CreateTablet for tablet 643bdb5fe6d24d44ac2c501bfb72e6ba (DEFAULT_TABLE table=test-table [id=5666e36076a44fdfa8b11c74935da38b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:33.500550 12310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 643bdb5fe6d24d44ac2c501bfb72e6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:33.500551 12698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 643bdb5fe6d24d44ac2c501bfb72e6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:33.501963 12501 tablet_service.cc:1467] Processing CreateTablet for tablet 643bdb5fe6d24d44ac2c501bfb72e6ba (DEFAULT_TABLE table=test-table [id=5666e36076a44fdfa8b11c74935da38b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 17:52:33.503252 12501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 643bdb5fe6d24d44ac2c501bfb72e6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:33.508176 12817 tablet_bootstrap.cc:492] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142: Bootstrap starting.
I20260812 17:52:33.509855 12818 tablet_bootstrap.cc:492] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8: Bootstrap starting.
I20260812 17:52:33.510680 12817 tablet_bootstrap.cc:654] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:33.511499 12817 log.cc:824] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:33.512058 12819 tablet_bootstrap.cc:492] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b: Bootstrap starting.
I20260812 17:52:33.512418 12818 tablet_bootstrap.cc:654] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:33.513371 12818 log.cc:824] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:33.514638 12819 tablet_bootstrap.cc:654] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:33.514839 12817 tablet_bootstrap.cc:492] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142: No bootstrap required, opened a new log
I20260812 17:52:33.515065 12817 ts_tablet_manager.cc:1397] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 17:52:33.515332 12818 tablet_bootstrap.cc:492] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8: No bootstrap required, opened a new log
I20260812 17:52:33.515601 12818 ts_tablet_manager.cc:1397] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260812 17:52:33.515604 12819 log.cc:824] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:33.517845 12819 tablet_bootstrap.cc:492] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b: No bootstrap required, opened a new log
I20260812 17:52:33.518080 12819 ts_tablet_manager.cc:1397] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 17:52:33.519403 12818 raft_consensus.cc:348] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c2307e9c54348fb933552cfffcb8142" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42611 } } peers { permanent_uuid: "037a8609ccfc4ce8b4173e26d425c84b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 46751 } } peers { permanent_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 42581 } }
I20260812 17:52:33.519786 12818 raft_consensus.cc:374] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:33.519850 12818 raft_consensus.cc:729] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2db2f8d625c4d109c8fa4dd17981fc8, State: Initialized, Role: FOLLOWER
I20260812 17:52:33.519758 12817 raft_consensus.cc:348] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c2307e9c54348fb933552cfffcb8142" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42611 } } peers { permanent_uuid: "037a8609ccfc4ce8b4173e26d425c84b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 46751 } } peers { permanent_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 42581 } }
I20260812 17:52:33.520422 12817 raft_consensus.cc:374] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:33.520504 12817 raft_consensus.cc:729] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c2307e9c54348fb933552cfffcb8142, State: Initialized, Role: FOLLOWER
I20260812 17:52:33.520440 12818 consensus_queue.cc:260] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c2307e9c54348fb933552cfffcb8142" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42611 } } peers { permanent_uuid: "037a8609ccfc4ce8b4173e26d425c84b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 46751 } } peers { permanent_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 42581 } }
I20260812 17:52:33.521024 12817 consensus_queue.cc:260] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c2307e9c54348fb933552cfffcb8142" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42611 } } peers { permanent_uuid: "037a8609ccfc4ce8b4173e26d425c84b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 46751 } } peers { permanent_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 42581 } }
I20260812 17:52:33.521225 12406 heartbeater.cc:503] Master 127.13.38.62:38187 was elected leader, sending a full tablet report...
I20260812 17:52:33.521639 12818 ts_tablet_manager.cc:1428] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 17:52:33.522070 12817 ts_tablet_manager.cc:1428] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 17:52:33.522233 12790 heartbeater.cc:503] Master 127.13.38.62:38187 was elected leader, sending a full tablet report...
I20260812 17:52:33.522675 12819 raft_consensus.cc:348] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c2307e9c54348fb933552cfffcb8142" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42611 } } peers { permanent_uuid: "037a8609ccfc4ce8b4173e26d425c84b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 46751 } } peers { permanent_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 42581 } }
I20260812 17:52:33.522955 12819 raft_consensus.cc:374] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:33.523042 12819 raft_consensus.cc:729] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 037a8609ccfc4ce8b4173e26d425c84b, State: Initialized, Role: FOLLOWER
I20260812 17:52:33.523500 12819 consensus_queue.cc:260] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c2307e9c54348fb933552cfffcb8142" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42611 } } peers { permanent_uuid: "037a8609ccfc4ce8b4173e26d425c84b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 46751 } } peers { permanent_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 42581 } }
I20260812 17:52:33.524262 12603 heartbeater.cc:503] Master 127.13.38.62:38187 was elected leader, sending a full tablet report...
I20260812 17:52:33.524655 12819 ts_tablet_manager.cc:1428] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 17:52:33.566083 12827 raft_consensus.cc:482] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 17:52:33.566373 12827 raft_consensus.cc:504] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c2307e9c54348fb933552cfffcb8142" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42611 } } peers { permanent_uuid: "037a8609ccfc4ce8b4173e26d425c84b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 46751 } } peers { permanent_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 42581 } }
I20260812 17:52:33.568030 12827 leader_election.cc:288] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c2307e9c54348fb933552cfffcb8142 (127.13.38.3:42611), 037a8609ccfc4ce8b4173e26d425c84b (127.13.38.2:46751)
I20260812 17:52:33.572928 12815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.568811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:53089 (local address 127.13.38.2:46751)
0812 17:52:33.568972 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:33.568980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:33.569000 (+ 20us) server_negotiation.cc:407] Connection header received
0812 17:52:33.569053 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:33.569056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:33.569106 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:33.569190 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:33.569595 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.570534 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.571916 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.572321 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.572451 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.572531 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:33.572718 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 17:52:33.572798 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 17:52:33.572958 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.568423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.2:46751 (local address 127.13.38.1:53089)
0812 17:52:33.568817 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:33.568832 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:33.568941 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:33.569223 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:33.569228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:33.569251 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:33.569436 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.569443 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.570700 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.570703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.571766 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.571777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.572458 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.572462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.572565 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.572594 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 17:52:33.572700 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 17:52:33.572779 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":4,"thread_start_us":116,"threads_started":1}
I20260812 17:52:33.573061 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.568424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.38.1:47827 (local address 127.13.38.3:42611)
0812 17:52:33.568601 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:33.568608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:33.568769 (+ 161us) server_negotiation.cc:407] Connection header received
0812 17:52:33.569032 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:33.569035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:33.569091 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:33.569150 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 17:52:33.569706 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.570536 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.571892 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.572337 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.572481 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.572551 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:33.572831 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 17:52:33.572929 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 17:52:33.573482 12831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.568257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.3:42611 (local address 127.13.38.1:47827)
0812 17:52:33.568642 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:33.568660 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:52:33.568870 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:52:33.569327 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:52:33.569330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:52:33.569348 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 17:52:33.569548 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.569556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.570714 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.570717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.571727 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:52:33.571740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.572489 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.572493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:52:33.572644 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.572679 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 17:52:33.573212 (+ 533us) client_negotiation.cc:240] Negotiation successful
0812 17:52:33.573299 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":72,"thread_start_us":173,"threads_started":1}
I20260812 17:52:33.573994 12531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "643bdb5fe6d24d44ac2c501bfb72e6ba" candidate_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "037a8609ccfc4ce8b4173e26d425c84b" is_pre_election: true
I20260812 17:52:33.574259 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "643bdb5fe6d24d44ac2c501bfb72e6ba" candidate_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c2307e9c54348fb933552cfffcb8142" is_pre_election: true
I20260812 17:52:33.574362 12531 raft_consensus.cc:2454] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2db2f8d625c4d109c8fa4dd17981fc8 in term 0.
I20260812 17:52:33.574590 12722 raft_consensus.cc:2454] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2db2f8d625c4d109c8fa4dd17981fc8 in term 0.
I20260812 17:52:33.574954 12243 leader_election.cc:302] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 037a8609ccfc4ce8b4173e26d425c84b, d2db2f8d625c4d109c8fa4dd17981fc8; no voters:
I20260812 17:52:33.575327 12827 raft_consensus.cc:2781] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 17:52:33.575453 12827 raft_consensus.cc:482] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 17:52:33.575568 12827 raft_consensus.cc:3037] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:33.576713 12827 raft_consensus.cc:504] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c2307e9c54348fb933552cfffcb8142" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42611 } } peers { permanent_uuid: "037a8609ccfc4ce8b4173e26d425c84b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 46751 } } peers { permanent_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 42581 } }
I20260812 17:52:33.577158 12827 leader_election.cc:288] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [CANDIDATE]: Term 1 election: Requested vote from peers 6c2307e9c54348fb933552cfffcb8142 (127.13.38.3:42611), 037a8609ccfc4ce8b4173e26d425c84b (127.13.38.2:46751)
I20260812 17:52:33.577608 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "643bdb5fe6d24d44ac2c501bfb72e6ba" candidate_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c2307e9c54348fb933552cfffcb8142"
I20260812 17:52:33.577699 12531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "643bdb5fe6d24d44ac2c501bfb72e6ba" candidate_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "037a8609ccfc4ce8b4173e26d425c84b"
I20260812 17:52:33.577762 12722 raft_consensus.cc:3037] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:33.577975 12531 raft_consensus.cc:3037] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:33.578971 12722 raft_consensus.cc:2454] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2db2f8d625c4d109c8fa4dd17981fc8 in term 1.
I20260812 17:52:33.579258 12531 raft_consensus.cc:2454] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2db2f8d625c4d109c8fa4dd17981fc8 in term 1.
I20260812 17:52:33.579445 12245 leader_election.cc:302] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c2307e9c54348fb933552cfffcb8142, d2db2f8d625c4d109c8fa4dd17981fc8; no voters:
I20260812 17:52:33.579763 12827 raft_consensus.cc:2781] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:33.580094 12827 raft_consensus.cc:686] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [term 1 LEADER]: Becoming Leader. State: Replica: d2db2f8d625c4d109c8fa4dd17981fc8, State: Running, Role: LEADER
I20260812 17:52:33.580493 12827 consensus_queue.cc:237] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c2307e9c54348fb933552cfffcb8142" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42611 } } peers { permanent_uuid: "037a8609ccfc4ce8b4173e26d425c84b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 46751 } } peers { permanent_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 42581 } }
I20260812 17:52:33.584563 12143 catalog_manager.cc:5495] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2db2f8d625c4d109c8fa4dd17981fc8 (127.13.38.1). New cstate: current_term: 1 leader_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c2307e9c54348fb933552cfffcb8142" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "037a8609ccfc4ce8b4173e26d425c84b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 46751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2db2f8d625c4d109c8fa4dd17981fc8" member_type: VOTER last_known_addr { host: "127.13.38.1" port: 42581 } health_report { overall_health: HEALTHY } } }
I20260812 17:52:33.598917 12816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.594974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57010 (local address 127.13.38.1:42581)
0812 17:52:33.595218 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:33.595222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:33.595237 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:52:33.595333 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:33.595335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:33.595392 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:33.595492 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:52:33.595904 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.596437 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.597444 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:33.597666 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:33.597755 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:33.598023 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:52:33.598495 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:52:33.598612 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:33.598730 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 17:52:33.598784 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260812 17:52:33.606021 12722 raft_consensus.cc:1261] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142 [term 1 FOLLOWER]: Refusing update from remote peer d2db2f8d625c4d109c8fa4dd17981fc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:52:33.606021 12531 raft_consensus.cc:1261] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b [term 1 FOLLOWER]: Refusing update from remote peer d2db2f8d625c4d109c8fa4dd17981fc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:52:33.606890 12833 consensus_queue.cc:1035] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c2307e9c54348fb933552cfffcb8142" member_type: VOTER last_known_addr { host: "127.13.38.3" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:52:33.607103 12827 consensus_queue.cc:1035] T 643bdb5fe6d24d44ac2c501bfb72e6ba P d2db2f8d625c4d109c8fa4dd17981fc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "037a8609ccfc4ce8b4173e26d425c84b" member_type: VOTER last_known_addr { host: "127.13.38.2" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:52:33.626945 12143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:52:33.627163 12143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 17:52:33.629948 12143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad238463d48f4e35abf380e5447d3808: Sending DeleteTablet for 3 replicas of tablet 643bdb5fe6d24d44ac2c501bfb72e6ba
I20260812 17:52:33.630962 12501 tablet_service.cc:1514] Processing DeleteTablet for tablet 643bdb5fe6d24d44ac2c501bfb72e6ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:33 UTC) from {username='slave'} at 127.0.0.1:40754
I20260812 17:52:33.631016 12698 tablet_service.cc:1514] Processing DeleteTablet for tablet 643bdb5fe6d24d44ac2c501bfb72e6ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:33 UTC) from {username='slave'} at 127.0.0.1:58270
I20260812 17:52:33.631369 12310 tablet_service.cc:1514] Processing DeleteTablet for tablet 643bdb5fe6d24d44ac2c501bfb72e6ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 17:52:33 UTC) from {username='slave'} at 127.0.0.1:56996
I20260812 17:52:33.631693 12847 tablet_replica.cc:330] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142: stopping tablet replica
I20260812 17:52:33.632125 12847 raft_consensus.cc:2229] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:52:33.632159 12845 tablet_replica.cc:330] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b: stopping tablet replica
I20260812 17:52:33.632479 12845 raft_consensus.cc:2229] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 17:52:33.632896 12845 raft_consensus.cc:2258] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:52:33.632959 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 12218
I20260812 17:52:33.632946 12847 raft_consensus.cc:2258] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 17:52:33.644536 12845 ts_tablet_manager.cc:1905] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:52:33.644650 12847 ts_tablet_manager.cc:1905] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 17:52:33.647486 12845 ts_tablet_manager.cc:1918] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:52:33.647737 12845 log.cc:1196] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-1/wal/wals/643bdb5fe6d24d44ac2c501bfb72e6ba
I20260812 17:52:33.648082 12845 ts_tablet_manager.cc:1939] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 037a8609ccfc4ce8b4173e26d425c84b: Deleting consensus metadata
I20260812 17:52:33.648919 12847 ts_tablet_manager.cc:1918] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 17:52:33.649066 12847 log.cc:1196] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142: Deleting WAL directory at /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786556880371321-13464-0/minicluster-data/ts-2/wal/wals/643bdb5fe6d24d44ac2c501bfb72e6ba
I20260812 17:52:33.649412 12847 ts_tablet_manager.cc:1939] T 643bdb5fe6d24d44ac2c501bfb72e6ba P 6c2307e9c54348fb933552cfffcb8142: Deleting consensus metadata
I20260812 17:52:33.649777 12118 catalog_manager.cc:4841] TS 037a8609ccfc4ce8b4173e26d425c84b (127.13.38.2:46751): tablet 643bdb5fe6d24d44ac2c501bfb72e6ba (table test-table [id=5666e36076a44fdfa8b11c74935da38b]) successfully deleted
I20260812 17:52:33.650915 12120 catalog_manager.cc:4841] TS 6c2307e9c54348fb933552cfffcb8142 (127.13.38.3:42611): tablet 643bdb5fe6d24d44ac2c501bfb72e6ba (table test-table [id=5666e36076a44fdfa8b11c74935da38b]) successfully deleted
W20260812 17:52:33.651427 12121 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.38.1:42581 (error 108)
I20260812 17:52:33.651719 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 12411
I20260812 17:52:33.670574 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:33.670184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.1:42581 (local address 127.0.0.1:57016)
0812 17:52:33.670353 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 17:52:33.670442 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.1:42581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260812 17:52:33.670823 12121 catalog_manager.cc:4584] TS d2db2f8d625c4d109c8fa4dd17981fc8 (127.13.38.1:42581): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 643bdb5fe6d24d44ac2c501bfb72e6ba: Network error: Client connection negotiation failed: client connection to 127.13.38.1:42581: connect: Connection refused (error 111)
I20260812 17:52:33.676712 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 12606
I20260812 17:52:33.697170 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 12098
2026-08-12T17:52:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1230 ms)
[----------] 6 tests from EncryptionPolicyTest (12312 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:33 dist-test-slave-sb2z krb5kdc[12852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:33 dist-test-slave-sb2z krb5kdc[12852](info): set up 2 sockets
Aug 12 17:52:33 dist-test-slave-sb2z krb5kdc[12852](info): commencing operation
krb5kdc: starting...
W20260812 17:52:35.018368 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:35 dist-test-slave-sb2z krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 17:52:35.168186 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:35.302053 12869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:35.302383 12869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:35.302481 12869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:35.307219 12869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:35.307299 12869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:35.307319 12869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:35.307336 12869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:35.307351 12869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:35.313138 12869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36099
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:35.314432 12869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:35.315663 12869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:35.327087 12875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:35.327088 12879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:35.327149 12876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:35.327610 12869 server_base.cc:1034] running on GCE node
I20260812 17:52:35.328294 12869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:35.329586 12869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:35.330819 12869 hybrid_clock.cc:648] HybridClock initialized: now 1786557155330797 us; error 43 us; skew 500 ppm
Aug 12 17:52:35 dist-test-slave-sb2z krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:35.334264 12869 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 17:52:35.335327 12869 webserver.cc:460] Webserver started at http://127.0.0.1:33147/ using document root <none> and password file <none>
I20260812 17:52:35.335994 12869 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:35.336086 12869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:35.336285 12869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:35.338557 12869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "7c4cf21739a04784b357a739237d4738"
format_stamp: "Formatted at 2026-08-12 17:52:35 on dist-test-slave-sb2z"
server_key: "0561592eb96094ff73d016b66146f2a4"
server_key_iv: "13327d5a07403091935679db6619a8bf"
server_key_version: "encryptionkey@0"
I20260812 17:52:35.339115 12869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "7c4cf21739a04784b357a739237d4738"
format_stamp: "Formatted at 2026-08-12 17:52:35 on dist-test-slave-sb2z"
server_key: "0561592eb96094ff73d016b66146f2a4"
server_key_iv: "13327d5a07403091935679db6619a8bf"
server_key_version: "encryptionkey@0"
I20260812 17:52:35.343293 12869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:52:35.346081 12886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:35.347754 12869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:35.347914 12869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "7c4cf21739a04784b357a739237d4738"
format_stamp: "Formatted at 2026-08-12 17:52:35 on dist-test-slave-sb2z"
server_key: "0561592eb96094ff73d016b66146f2a4"
server_key_iv: "13327d5a07403091935679db6619a8bf"
server_key_version: "encryptionkey@0"
I20260812 17:52:35.348052 12869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:35.367512 12869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:35.371044 12869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:35.371309 12869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:35.379448 12869 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36099
I20260812 17:52:35.379455 12973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36099 every 8 connection(s)
I20260812 17:52:35.380555 12869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:35.383587 12974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:35.387717 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 12869
I20260812 17:52:35.387952 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:35.388273 13464 external_mini_cluster.cc:1428] Setting key 2f4b7304934abed559fa3c9c4b6cd88e
I20260812 17:52:35.390061 12974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738: Bootstrap starting.
I20260812 17:52:35.392490 12974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:35.393275 12974 log.cc:824] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:35.395249 12974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738: No bootstrap required, opened a new log
Aug 12 17:52:35 dist-test-slave-sb2z krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 17:52:35.399334 12974 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c4cf21739a04784b357a739237d4738" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36099 } }
I20260812 17:52:35.399680 12974 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:35.399770 12974 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c4cf21739a04784b357a739237d4738, State: Initialized, Role: FOLLOWER
I20260812 17:52:35.400341 12974 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c4cf21739a04784b357a739237d4738" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36099 } }
I20260812 17:52:35.400485 12974 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:35.400578 12974 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:35.400707 12974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:35.401690 12974 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c4cf21739a04784b357a739237d4738" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36099 } }
I20260812 17:52:35.402068 12974 leader_election.cc:302] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c4cf21739a04784b357a739237d4738; no voters:
I20260812 17:52:35.402402 12974 leader_election.cc:288] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:35.402527 12981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:35.402776 12981 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [term 1 LEADER]: Becoming Leader. State: Replica: 7c4cf21739a04784b357a739237d4738, State: Running, Role: LEADER
I20260812 17:52:35.403116 12981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c4cf21739a04784b357a739237d4738" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36099 } }
I20260812 17:52:35.403923 12974 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:35.404742 12985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c4cf21739a04784b357a739237d4738" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c4cf21739a04784b357a739237d4738" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36099 } } }
I20260812 17:52:35.404860 12985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:35.405056 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:35.389862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33119 (local address 127.0.0.1:36099)
0812 17:52:35.390420 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:35.390438 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:35.390476 (+ 38us) server_negotiation.cc:407] Connection header received
0812 17:52:35.391154 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:35.391172 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:35.391635 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:35.392002 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:35.393004 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:35.394101 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:35.394986 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:35.395574 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:35.395751 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:35.398618 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:35.398624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:35.398640 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:35.398670 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:35.400795 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:35.401428 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:35.401435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:35.401442 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:35.401571 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:35.402009 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:35.402012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:35.402014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:35.402609 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:35.402834 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:35.403670 (+ 836us) server_negotiation.cc:299] Negotiation successful
0812 17:52:35.403914 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":152,"threads_started":1}
I20260812 17:52:35.404314 12981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c4cf21739a04784b357a739237d4738. Latest consensus state: current_term: 1 leader_uuid: "7c4cf21739a04784b357a739237d4738" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c4cf21739a04784b357a739237d4738" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36099 } } }
I20260812 17:52:35.405428 12981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:35.406193 12995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:35.408682 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:35.410153 12995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:35.417003 12995 catalog_manager.cc:1347] Generated new cluster ID: a3b0a0ca952546568bbec8a38484b34b
I20260812 17:52:35.417104 12995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:35.422209 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:35.411063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56343 (local address 127.0.0.1:36099)
0812 17:52:35.411184 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:35.411189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:35.411844 (+ 655us) server_negotiation.cc:407] Connection header received
0812 17:52:35.411981 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:35.411984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:35.412036 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:35.412132 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:35.413093 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:35.413873 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:35.415396 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:35.415787 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:35.415903 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:35.418139 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:35.418143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:35.418146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:35.418175 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:35.420070 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:35.420737 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:35.420740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:35.420742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:35.420831 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:35.421243 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:35.421246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:35.421248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:35.421454 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:35.421583 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:35.421937 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 17:52:35.422041 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 17:52:35.441769 12995 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:35.443127 12995 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:35.450373 12995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c4cf21739a04784b357a739237d4738: Generated new TSK 0
I20260812 17:52:35.451313 12995 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:35.459815 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 12869
W20260812 17:52:35.475898 12850 connection.cc:535] client connection to 127.0.0.1:36099 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36099): Connection reset by peer (error 104)
2026-08-12T17:52:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1763 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:35 dist-test-slave-sb2z krb5kdc[13015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:35 dist-test-slave-sb2z krb5kdc[13015](info): set up 2 sockets
Aug 12 17:52:35 dist-test-slave-sb2z krb5kdc[13015](info): commencing operation
krb5kdc: starting...
W20260812 17:52:36.801604 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:36 dist-test-slave-sb2z krb5kdc[13015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 17:52:36.956964 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42047
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:37.094884 13032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:37.095353 13032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:37.095456 13032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:37.100776 13032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:37.100920 13032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:37.100962 13032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:37.101003 13032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:37.101038 13032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:37.107055 13032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42047
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:37.108464 13032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:37.109637 13032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:37.119793 13039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:37.119783 13042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:37.119788 13038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:37.119992 13032 server_base.cc:1034] running on GCE node
I20260812 17:52:37.120807 13032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:37.122205 13032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:37.123435 13032 hybrid_clock.cc:648] HybridClock initialized: now 1786557157123416 us; error 40 us; skew 500 ppm
Aug 12 17:52:37 dist-test-slave-sb2z krb5kdc[13015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:37.126844 13032 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 17:52:37.128046 13032 webserver.cc:460] Webserver started at http://127.0.0.1:40119/ using document root <none> and password file <none>
I20260812 17:52:37.128700 13032 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:37.128805 13032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:37.129074 13032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:37.131414 13032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "7eace754160a45ae8e3c3f7618554e7c"
format_stamp: "Formatted at 2026-08-12 17:52:37 on dist-test-slave-sb2z"
server_key: "3a471bdd902fa6fc48927b9ccb81a764"
server_key_iv: "cfa840bd5bfa71b4da42e01760e28547"
server_key_version: "encryptionkey@0"
I20260812 17:52:37.132187 13032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "7eace754160a45ae8e3c3f7618554e7c"
format_stamp: "Formatted at 2026-08-12 17:52:37 on dist-test-slave-sb2z"
server_key: "3a471bdd902fa6fc48927b9ccb81a764"
server_key_iv: "cfa840bd5bfa71b4da42e01760e28547"
server_key_version: "encryptionkey@0"
I20260812 17:52:37.136976 13032 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:52:37.140255 13051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:37.141981 13032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 17:52:37.142241 13032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "7eace754160a45ae8e3c3f7618554e7c"
format_stamp: "Formatted at 2026-08-12 17:52:37 on dist-test-slave-sb2z"
server_key: "3a471bdd902fa6fc48927b9ccb81a764"
server_key_iv: "cfa840bd5bfa71b4da42e01760e28547"
server_key_version: "encryptionkey@0"
I20260812 17:52:37.142400 13032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:37.164364 13032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:37.168381 13032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:37.168648 13032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:37.177096 13032 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42047
I20260812 17:52:37.177140 13129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42047 every 8 connection(s)
I20260812 17:52:37.178247 13032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:37.181872 13131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:37.187270 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 13032
I20260812 17:52:37.187494 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:37.187893 13464 external_mini_cluster.cc:1428] Setting key 106d31f7ba058cd662b851b6e1ab8d4e
I20260812 17:52:37.188542 13131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c: Bootstrap starting.
I20260812 17:52:37.191614 13131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:37.192517 13131 log.cc:824] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:37.194780 13131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c: No bootstrap required, opened a new log
I20260812 17:52:37.198253 13136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:37.189520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48481 (local address 127.0.0.1:42047)
0812 17:52:37.190157 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:37.190176 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:37.190219 (+ 43us) server_negotiation.cc:407] Connection header received
0812 17:52:37.190954 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:37.190973 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:37.191425 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:37.191857 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:37.192322 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:37.193385 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:37.194389 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:37.194874 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:37.195058 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:37.196232 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:37.196237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:37.196252 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:37.196702 (+ 450us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:37.196760 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:37.196784 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:37.196894 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:37.197248 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 17:52:37.197415 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":164,"threads_started":1}
I20260812 17:52:37.199218 13131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eace754160a45ae8e3c3f7618554e7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } }
I20260812 17:52:37.199504 13131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:37.199591 13131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eace754160a45ae8e3c3f7618554e7c, State: Initialized, Role: FOLLOWER
I20260812 17:52:37.200156 13131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eace754160a45ae8e3c3f7618554e7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } }
I20260812 17:52:37.200297 13131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:37.200384 13131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:37.200502 13131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:37.201601 13131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eace754160a45ae8e3c3f7618554e7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } }
I20260812 17:52:37.201725 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:37.202060 13131 leader_election.cc:302] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7eace754160a45ae8e3c3f7618554e7c; no voters:
I20260812 17:52:37.202513 13131 leader_election.cc:288] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:37.202764 13138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:37.203029 13138 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [term 1 LEADER]: Becoming Leader. State: Replica: 7eace754160a45ae8e3c3f7618554e7c, State: Running, Role: LEADER
I20260812 17:52:37.203424 13138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eace754160a45ae8e3c3f7618554e7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } }
I20260812 17:52:37.204396 13131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:37.204811 13138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7eace754160a45ae8e3c3f7618554e7c. Latest consensus state: current_term: 1 leader_uuid: "7eace754160a45ae8e3c3f7618554e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eace754160a45ae8e3c3f7618554e7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } } }
I20260812 17:52:37.204947 13138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:37.205322 13141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7eace754160a45ae8e3c3f7618554e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eace754160a45ae8e3c3f7618554e7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } } }
I20260812 17:52:37.205391 13149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:37.205433 13141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:37.209021 13149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:37.213256 13136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:37.204618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45515 (local address 127.0.0.1:42047)
0812 17:52:37.204737 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:37.204742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:37.206178 (+ 1436us) server_negotiation.cc:407] Connection header received
0812 17:52:37.206390 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:37.206393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:37.206444 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:37.206498 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:37.207303 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:37.208119 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:37.210160 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:37.210544 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:37.210660 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:37.212617 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:37.212622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:37.212625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:37.212650 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:37.212683 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:37.212697 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:37.212816 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:37.213015 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 17:52:37.213061 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 17:52:37.217051 13149 catalog_manager.cc:1347] Generated new cluster ID: 19ffc3a74b854befa47a672193408bab
I20260812 17:52:37.217161 13149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:37.247445 13149 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:37.248818 13149 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:37.258342 13149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7eace754160a45ae8e3c3f7618554e7c: Generated new TSK 0
I20260812 17:52:37.259240 13149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:37.271869 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 13032
W20260812 17:52:37.285449 13012 connection.cc:535] client connection to 127.0.0.1:42047 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42047): Connection reset by peer (error 104)
2026-08-12T17:52:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1809 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:37 dist-test-slave-sb2z krb5kdc[13175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:37 dist-test-slave-sb2z krb5kdc[13175](info): set up 2 sockets
Aug 12 17:52:37 dist-test-slave-sb2z krb5kdc[13175](info): commencing operation
krb5kdc: starting...
W20260812 17:52:38.620859 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:38 dist-test-slave-sb2z krb5kdc[13175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 17:52:38.776749 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:38.902660 13192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:38.903020 13192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:38.903156 13192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:38.908092 13192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:38.908182 13192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:38.908205 13192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:38.908222 13192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:38.908238 13192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:38.914265 13192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33977
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:38.915645 13192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:38.916810 13192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:38.926731 13198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:38.926795 13192 server_base.cc:1034] running on GCE node
W20260812 17:52:38.926721 13201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:38.926731 13199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:38.927861 13192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:38.929154 13192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:38.930361 13192 hybrid_clock.cc:648] HybridClock initialized: now 1786557158930346 us; error 32 us; skew 500 ppm
Aug 12 17:52:38 dist-test-slave-sb2z krb5kdc[13175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557158, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:38.937438 13192 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 17:52:38.938565 13192 webserver.cc:460] Webserver started at http://127.0.0.1:44803/ using document root <none> and password file <none>
I20260812 17:52:38.939241 13192 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:38.939352 13192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:38.939719 13192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:38.942139 13192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "055435205a7646cf90331322c67a3671"
format_stamp: "Formatted at 2026-08-12 17:52:38 on dist-test-slave-sb2z"
server_key: "d098dd3dc75cad08dd6a193dd05ef62a"
server_key_iv: "1cc5375783cb563a9cb296cae592bdb3"
server_key_version: "encryptionkey@0"
I20260812 17:52:38.942870 13192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "055435205a7646cf90331322c67a3671"
format_stamp: "Formatted at 2026-08-12 17:52:38 on dist-test-slave-sb2z"
server_key: "d098dd3dc75cad08dd6a193dd05ef62a"
server_key_iv: "1cc5375783cb563a9cb296cae592bdb3"
server_key_version: "encryptionkey@0"
I20260812 17:52:38.948487 13192 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260812 17:52:38.952670 13212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:38.954558 13192 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:52:38.954880 13192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "055435205a7646cf90331322c67a3671"
format_stamp: "Formatted at 2026-08-12 17:52:38 on dist-test-slave-sb2z"
server_key: "d098dd3dc75cad08dd6a193dd05ef62a"
server_key_iv: "1cc5375783cb563a9cb296cae592bdb3"
server_key_version: "encryptionkey@0"
I20260812 17:52:38.955044 13192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:38.960397 13192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:38.964184 13192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:38.964445 13192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:38.972607 13192 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33977
I20260812 17:52:38.972615 13288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33977 every 8 connection(s)
I20260812 17:52:38.973698 13192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:38.975914 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 13192
I20260812 17:52:38.976137 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:38.976467 13464 external_mini_cluster.cc:1428] Setting key fab2f717ed768722f7403317fa74dc00
I20260812 17:52:38.977392 13290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:38.982281 13295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:38.978035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39317 (local address 127.0.0.1:33977)
0812 17:52:38.978593 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:38.978609 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:38.978649 (+ 40us) server_negotiation.cc:407] Connection header received
0812 17:52:38.979347 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:38.979364 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:38.979704 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:38.980099 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:38.981001 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:38.981006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:38.981020 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:38.981032 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:38.981091 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:38.981111 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:38.981273 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:38.981350 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 17:52:38.981534 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":162,"threads_started":1}
I20260812 17:52:38.984522 13290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671: Bootstrap starting.
I20260812 17:52:38.985602 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:38.987102 13290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:38.988090 13290 log.cc:824] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:38.990418 13295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:38.987994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33573 (local address 127.0.0.1:33977)
0812 17:52:38.988150 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:38.988155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:38.988450 (+ 295us) server_negotiation.cc:407] Connection header received
0812 17:52:38.988641 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:38.988646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:38.988691 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:38.988796 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:38.989799 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:38.989802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:38.989804 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:38.989815 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:38.989842 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:38.989853 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:38.989941 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:38.990152 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 17:52:38.990247 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:52:38.991315 13290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671: No bootstrap required, opened a new log
I20260812 17:52:38.995859 13290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055435205a7646cf90331322c67a3671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33977 } }
I20260812 17:52:38.996287 13290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:38.996362 13290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055435205a7646cf90331322c67a3671, State: Initialized, Role: FOLLOWER
I20260812 17:52:38.996845 13290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055435205a7646cf90331322c67a3671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33977 } }
I20260812 17:52:38.996982 13290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:38.997030 13290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:38.997167 13290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:38.998245 13290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055435205a7646cf90331322c67a3671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33977 } }
I20260812 17:52:38.998852 13290 leader_election.cc:302] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 055435205a7646cf90331322c67a3671; no voters:
I20260812 17:52:38.999312 13290 leader_election.cc:288] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:38.999665 13305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:38.999989 13305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [term 1 LEADER]: Becoming Leader. State: Replica: 055435205a7646cf90331322c67a3671, State: Running, Role: LEADER
I20260812 17:52:39.000347 13305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055435205a7646cf90331322c67a3671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33977 } }
I20260812 17:52:39.000944 13290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:39.002574 13309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "055435205a7646cf90331322c67a3671" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055435205a7646cf90331322c67a3671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33977 } } }
I20260812 17:52:39.002775 13309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:39.003188 13310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 055435205a7646cf90331322c67a3671. Latest consensus state: current_term: 1 leader_uuid: "055435205a7646cf90331322c67a3671" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055435205a7646cf90331322c67a3671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33977 } } }
I20260812 17:52:39.003320 13310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:39.003242 13329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:39.007192 13329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:39.014335 13329 catalog_manager.cc:1347] Generated new cluster ID: 124d8df98de04622b14e62a22f563d41
I20260812 17:52:39.014463 13329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:39.027938 13329 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:39.028469 13329 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:39.039170 13329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 055435205a7646cf90331322c67a3671: Generated new TSK 0
I20260812 17:52:39.040257 13329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:39.043742 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 13192
2026-08-12T17:52:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1775 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:39 dist-test-slave-sb2z krb5kdc[13341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:39 dist-test-slave-sb2z krb5kdc[13341](info): set up 2 sockets
Aug 12 17:52:39 dist-test-slave-sb2z krb5kdc[13341](info): commencing operation
krb5kdc: starting...
W20260812 17:52:40.367022 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:40 dist-test-slave-sb2z krb5kdc[13341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 17:52:40.516178 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33569
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:40.643258 13361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:40.643740 13361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:40.643836 13361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:40.648128 13361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:40.648262 13361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:40.648298 13361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:40.648340 13361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:40.648376 13361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:40.654464 13361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:40.655825 13361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:40.657032 13361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:40.666963 13368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:40.666967 13366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:40.666963 13370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:40.667125 13361 server_base.cc:1034] running on GCE node
I20260812 17:52:40.668187 13361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:40.669711 13361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:40.670979 13361 hybrid_clock.cc:648] HybridClock initialized: now 1786557160670961 us; error 45 us; skew 500 ppm
Aug 12 17:52:40 dist-test-slave-sb2z krb5kdc[13341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557160, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:40.674654 13361 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 17:52:40.675855 13361 webserver.cc:460] Webserver started at http://127.0.0.1:43947/ using document root <none> and password file <none>
I20260812 17:52:40.676594 13361 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:40.676698 13361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:40.676981 13361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:40.679499 13361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "7ca2c9f28a2b4bbca75689f6a00d8682"
format_stamp: "Formatted at 2026-08-12 17:52:40 on dist-test-slave-sb2z"
server_key: "98c9b9a527015c753368eda373b569f6"
server_key_iv: "8c7ecf02313e62f083d77903b93e8df9"
server_key_version: "encryptionkey@0"
I20260812 17:52:40.680466 13361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "7ca2c9f28a2b4bbca75689f6a00d8682"
format_stamp: "Formatted at 2026-08-12 17:52:40 on dist-test-slave-sb2z"
server_key: "98c9b9a527015c753368eda373b569f6"
server_key_iv: "8c7ecf02313e62f083d77903b93e8df9"
server_key_version: "encryptionkey@0"
I20260812 17:52:40.684930 13361 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:52:40.687875 13378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:40.689342 13361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 17:52:40.689478 13361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "7ca2c9f28a2b4bbca75689f6a00d8682"
format_stamp: "Formatted at 2026-08-12 17:52:40 on dist-test-slave-sb2z"
server_key: "98c9b9a527015c753368eda373b569f6"
server_key_iv: "8c7ecf02313e62f083d77903b93e8df9"
server_key_version: "encryptionkey@0"
I20260812 17:52:40.689595 13361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:40.718473 13361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:40.722775 13361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:40.723026 13361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:40.733520 13361 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33569
I20260812 17:52:40.733548 13447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33569 every 8 connection(s)
I20260812 17:52:40.734649 13361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:40.736088 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 13361
I20260812 17:52:40.736255 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:40.736610 13464 external_mini_cluster.cc:1428] Setting key b2e3938f0d2b765f1942c789599f43dc
I20260812 17:52:40.738559 13448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:40.745215 13448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682: Bootstrap starting.
Aug 12 17:52:40 dist-test-slave-sb2z krb5kdc[13341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 17:52:40.748272 13448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:40.749311 13448 log.cc:824] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:40.752127 13448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682: No bootstrap required, opened a new log
I20260812 17:52:40.753530 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:40.738237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42871 (local address 127.0.0.1:33569)
0812 17:52:40.738853 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:40.738871 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:40.738912 (+ 41us) server_negotiation.cc:407] Connection header received
0812 17:52:40.739685 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:40.739706 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:40.740098 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:40.740477 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:40.741428 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:40.742462 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:40.743394 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:40.743996 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:40.744196 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:40.747472 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:40.747479 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:40.747495 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:40.747589 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:40.749948 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:40.750611 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:40.750617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:40.750623 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:40.750761 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:40.751130 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:40.751133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:40.751134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:40.751673 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:40.751830 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:40.752343 (+ 513us) server_negotiation.cc:299] Negotiation successful
0812 17:52:40.752713 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":185,"threads_started":1}
I20260812 17:52:40.756672 13448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca2c9f28a2b4bbca75689f6a00d8682" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } }
I20260812 17:52:40.757071 13448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:40.757126 13448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca2c9f28a2b4bbca75689f6a00d8682, State: Initialized, Role: FOLLOWER
I20260812 17:52:40.757422 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:40.757793 13448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca2c9f28a2b4bbca75689f6a00d8682" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } }
I20260812 17:52:40.758020 13448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:40.758100 13448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:40.758209 13448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:40.759979 13448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca2c9f28a2b4bbca75689f6a00d8682" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } }
I20260812 17:52:40.760486 13448 leader_election.cc:302] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ca2c9f28a2b4bbca75689f6a00d8682; no voters:
I20260812 17:52:40.760948 13448 leader_election.cc:288] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:40.761036 13474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:40.761379 13474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [term 1 LEADER]: Becoming Leader. State: Replica: 7ca2c9f28a2b4bbca75689f6a00d8682, State: Running, Role: LEADER
I20260812 17:52:40.761792 13474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca2c9f28a2b4bbca75689f6a00d8682" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } }
I20260812 17:52:40.762689 13448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:40.765714 13477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ca2c9f28a2b4bbca75689f6a00d8682. Latest consensus state: current_term: 1 leader_uuid: "7ca2c9f28a2b4bbca75689f6a00d8682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca2c9f28a2b4bbca75689f6a00d8682" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } } }
I20260812 17:52:40.765688 13476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ca2c9f28a2b4bbca75689f6a00d8682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca2c9f28a2b4bbca75689f6a00d8682" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } } }
I20260812 17:52:40.765879 13477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:40.765951 13476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:40.766269 13498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:40.769999 13498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:40.770412 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:40.760481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43747 (local address 127.0.0.1:33569)
0812 17:52:40.760644 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:40.760649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:40.760868 (+ 219us) server_negotiation.cc:407] Connection header received
0812 17:52:40.761144 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:40.761148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:40.761211 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:40.761311 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:40.762279 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:40.763095 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:40.764424 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:40.764822 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:40.764928 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:40.766656 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:40.766663 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:40.766668 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:40.766699 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:40.768488 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:40.768983 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:40.768987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:40.768989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:40.769087 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:40.769405 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:40.769409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:40.769410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:40.769628 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:40.769777 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:40.770042 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 17:52:40.770238 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 17:52:40.777103 13498 catalog_manager.cc:1347] Generated new cluster ID: 1e1a573e5e98461ebba0a45166cae0ae
I20260812 17:52:40.777215 13498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:40.784497 13498 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:40.785607 13498 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:40.791213 13498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ca2c9f28a2b4bbca75689f6a00d8682: Generated new TSK 0
I20260812 17:52:40.792181 13498 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:40.797840 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 13361
2026-08-12T17:52:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1746 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:40 dist-test-slave-sb2z krb5kdc[13506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:40 dist-test-slave-sb2z krb5kdc[13506](info): set up 2 sockets
Aug 12 17:52:40 dist-test-slave-sb2z krb5kdc[13506](info): commencing operation
krb5kdc: starting...
W20260812 17:52:42.079800 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.006s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:42 dist-test-slave-sb2z krb5kdc[13506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 17:52:42.232008 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43363
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:42.357151 13523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:42.357477 13523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:42.357522 13523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:42.361802 13523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:42.361891 13523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:42.361912 13523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:42.361929 13523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:42.361945 13523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:42.367707 13523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:42.368830 13523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:42.370078 13523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:42.382753 13530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:42.382814 13531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:42.382773 13535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:42.383101 13523 server_base.cc:1034] running on GCE node
I20260812 17:52:42.383988 13523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:42.385255 13523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:42.386451 13523 hybrid_clock.cc:648] HybridClock initialized: now 1786557162386425 us; error 49 us; skew 500 ppm
Aug 12 17:52:42 dist-test-slave-sb2z krb5kdc[13506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557162, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:42.390095 13523 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 17:52:42.391189 13523 webserver.cc:460] Webserver started at http://127.0.0.1:42947/ using document root <none> and password file <none>
I20260812 17:52:42.391857 13523 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:42.391963 13523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:42.392206 13523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:42.394335 13523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "aee0ff184ef34bed838bd19a4f3b4a94"
format_stamp: "Formatted at 2026-08-12 17:52:42 on dist-test-slave-sb2z"
server_key: "2a66ea4a88e9ce9e263421d5940a705c"
server_key_iv: "11679a31a40c7254ea2e71f98c77eaef"
server_key_version: "encryptionkey@0"
I20260812 17:52:42.394927 13523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "aee0ff184ef34bed838bd19a4f3b4a94"
format_stamp: "Formatted at 2026-08-12 17:52:42 on dist-test-slave-sb2z"
server_key: "2a66ea4a88e9ce9e263421d5940a705c"
server_key_iv: "11679a31a40c7254ea2e71f98c77eaef"
server_key_version: "encryptionkey@0"
I20260812 17:52:42.399668 13523 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 17:52:42.402560 13543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:42.404168 13523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:42.404340 13523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "aee0ff184ef34bed838bd19a4f3b4a94"
format_stamp: "Formatted at 2026-08-12 17:52:42 on dist-test-slave-sb2z"
server_key: "2a66ea4a88e9ce9e263421d5940a705c"
server_key_iv: "11679a31a40c7254ea2e71f98c77eaef"
server_key_version: "encryptionkey@0"
I20260812 17:52:42.404474 13523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:42.418391 13523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:42.423056 13523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:42.423286 13523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:42.433111 13523 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43363
I20260812 17:52:42.433138 13626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43363 every 8 connection(s)
I20260812 17:52:42.434360 13523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:42.437839 13627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:42.441247 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 13523
I20260812 17:52:42.441423 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:42.441746 13464 external_mini_cluster.cc:1428] Setting key 004cc060a2c3e4b40c1e0bffbe205a76
I20260812 17:52:42.445245 13627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94: Bootstrap starting.
I20260812 17:52:42.447969 13627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:42.448920 13627 log.cc:824] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:42.451725 13627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94: No bootstrap required, opened a new log
I20260812 17:52:42.454326 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:42.443390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49235 (local address 127.0.0.1:43363)
0812 17:52:42.443993 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:42.444017 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:42.444071 (+ 54us) server_negotiation.cc:407] Connection header received
0812 17:52:42.445161 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:42.445191 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:42.445731 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:42.446192 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:42.446686 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:42.448463 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:42.449837 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:42.450538 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:42.450721 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:42.451803 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:42.451810 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:42.451829 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:42.452380 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:42.452454 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:42.452485 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:42.452591 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:42.452776 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 17:52:42.453016 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":143,"threads_started":1}
I20260812 17:52:42.455945 13627 raft_consensus.cc:348] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee0ff184ef34bed838bd19a4f3b4a94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43363 } }
I20260812 17:52:42.456223 13627 raft_consensus.cc:374] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:42.456286 13627 raft_consensus.cc:729] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aee0ff184ef34bed838bd19a4f3b4a94, State: Initialized, Role: FOLLOWER
I20260812 17:52:42.456830 13627 consensus_queue.cc:260] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee0ff184ef34bed838bd19a4f3b4a94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43363 } }
I20260812 17:52:42.457026 13627 raft_consensus.cc:388] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:42.457104 13627 raft_consensus.cc:482] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:42.457211 13627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:42.457899 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:42.458551 13627 raft_consensus.cc:504] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee0ff184ef34bed838bd19a4f3b4a94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43363 } }
I20260812 17:52:42.458953 13627 leader_election.cc:302] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aee0ff184ef34bed838bd19a4f3b4a94; no voters:
I20260812 17:52:42.459331 13627 leader_election.cc:288] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:42.459646 13637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:42.459940 13637 raft_consensus.cc:686] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [term 1 LEADER]: Becoming Leader. State: Replica: aee0ff184ef34bed838bd19a4f3b4a94, State: Running, Role: LEADER
I20260812 17:52:42.460403 13637 consensus_queue.cc:237] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee0ff184ef34bed838bd19a4f3b4a94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43363 } }
I20260812 17:52:42.461112 13627 sys_catalog.cc:564] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:42.462407 13644 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aee0ff184ef34bed838bd19a4f3b4a94. Latest consensus state: current_term: 1 leader_uuid: "aee0ff184ef34bed838bd19a4f3b4a94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee0ff184ef34bed838bd19a4f3b4a94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43363 } } }
I20260812 17:52:42.462555 13644 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:42.462903 13642 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aee0ff184ef34bed838bd19a4f3b4a94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee0ff184ef34bed838bd19a4f3b4a94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43363 } } }
I20260812 17:52:42.462999 13642 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:42.464068 13657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:42.466141 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:42.460810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59933 (local address 127.0.0.1:43363)
0812 17:52:42.460936 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:42.460940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:42.460956 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:52:42.461008 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:42.461011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:42.461061 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:42.461153 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:42.461614 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:42.462467 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:42.464128 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:42.464447 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:42.464526 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:42.465644 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:42.465648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:42.465651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:42.465666 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:42.465696 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:42.465710 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:42.465754 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:42.465926 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 17:52:42.465989 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":30}
I20260812 17:52:42.468079 13657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:42.474406 13657 catalog_manager.cc:1347] Generated new cluster ID: f6458b293faf488395841cf835c72ec3
I20260812 17:52:42.474504 13657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:42.496038 13657 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:42.497416 13657 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:42.502990 13657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aee0ff184ef34bed838bd19a4f3b4a94: Generated new TSK 0
I20260812 17:52:42.503908 13657 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:42.512740 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 13523
2026-08-12T17:52:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1723 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:42 dist-test-slave-sb2z krb5kdc[13678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:42 dist-test-slave-sb2z krb5kdc[13678](info): set up 2 sockets
Aug 12 17:52:42 dist-test-slave-sb2z krb5kdc[13678](info): commencing operation
krb5kdc: starting...
W20260812 17:52:43.796806 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:43 dist-test-slave-sb2z krb5kdc[13678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 17:52:43.940137 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36237
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:44.069044 13701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:44.069373 13701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:44.069449 13701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:44.073673 13701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:44.073779 13701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:44.073819 13701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:44.073889 13701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:44.073927 13701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:44.079603 13701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:44.080819 13701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:44.081930 13701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 17:52:44.091079 13701 server_base.cc:1034] running on GCE node
W20260812 17:52:44.090989 13712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:44.090981 13714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:44.090994 13710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:44.092093 13701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:44.093264 13701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:44.094468 13701 hybrid_clock.cc:648] HybridClock initialized: now 1786557164094435 us; error 50 us; skew 500 ppm
Aug 12 17:52:44 dist-test-slave-sb2z krb5kdc[13678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557164, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:44.101120 13701 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 17:52:44.102120 13701 webserver.cc:460] Webserver started at http://127.0.0.1:39601/ using document root <none> and password file <none>
I20260812 17:52:44.102726 13701 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:44.102826 13701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:44.103096 13701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:44.105386 13701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "2fc2bbcc98b54e33a3469c60c9633348"
format_stamp: "Formatted at 2026-08-12 17:52:44 on dist-test-slave-sb2z"
server_key: "6fb0c339deaa1550a69f674a6eb66fb8"
server_key_iv: "d634bf643ed7b5c2bae8214d319a110c"
server_key_version: "encryptionkey@0"
I20260812 17:52:44.105995 13701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "2fc2bbcc98b54e33a3469c60c9633348"
format_stamp: "Formatted at 2026-08-12 17:52:44 on dist-test-slave-sb2z"
server_key: "6fb0c339deaa1550a69f674a6eb66fb8"
server_key_iv: "d634bf643ed7b5c2bae8214d319a110c"
server_key_version: "encryptionkey@0"
I20260812 17:52:44.110255 13701 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 17:52:44.113232 13726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:44.114722 13701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 17:52:44.114859 13701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "2fc2bbcc98b54e33a3469c60c9633348"
format_stamp: "Formatted at 2026-08-12 17:52:44 on dist-test-slave-sb2z"
server_key: "6fb0c339deaa1550a69f674a6eb66fb8"
server_key_iv: "d634bf643ed7b5c2bae8214d319a110c"
server_key_version: "encryptionkey@0"
I20260812 17:52:44.114967 13701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:44.119946 13701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:44.123515 13701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:44.123767 13701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:44.132496 13701 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36237
I20260812 17:52:44.132526 13811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36237 every 8 connection(s)
I20260812 17:52:44.133561 13701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:44.136755 13813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:44.138967 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 13701
I20260812 17:52:44.139137 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:44.139434 13464 external_mini_cluster.cc:1428] Setting key 459ae913f4803f7a8cb54d60449c4592
I20260812 17:52:44.144065 13813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348: Bootstrap starting.
I20260812 17:52:44.146116 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:44.141103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56507 (local address 127.0.0.1:36237)
0812 17:52:44.141627 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:44.141650 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:44.141698 (+ 48us) server_negotiation.cc:407] Connection header received
0812 17:52:44.142773 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:44.142802 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:44.143203 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:44.143650 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:44.144566 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:44.144571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:44.144587 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:44.144600 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:44.144667 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:44.144695 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:44.144838 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:44.144907 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 17:52:44.145060 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":163,"threads_started":1}
I20260812 17:52:44.147063 13813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:44.147938 13813 log.cc:824] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:44.149437 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:44.150288 13813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348: No bootstrap required, opened a new log
I20260812 17:52:44.153648 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:44.151693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55033 (local address 127.0.0.1:36237)
0812 17:52:44.151830 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:44.151835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:44.152009 (+ 174us) server_negotiation.cc:407] Connection header received
0812 17:52:44.152199 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:44.152202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:44.152249 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:44.152341 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:44.153197 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:44.153201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:44.153204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:44.153217 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:44.153248 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:44.153269 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:44.153376 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:44.153464 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 17:52:44.153512 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 17:52:44.154717 13813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc2bbcc98b54e33a3469c60c9633348" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36237 } }
I20260812 17:52:44.155033 13813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:44.155122 13813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc2bbcc98b54e33a3469c60c9633348, State: Initialized, Role: FOLLOWER
I20260812 17:52:44.155673 13813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc2bbcc98b54e33a3469c60c9633348" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36237 } }
I20260812 17:52:44.155813 13813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:44.155923 13813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:44.156040 13813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:44.157088 13813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc2bbcc98b54e33a3469c60c9633348" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36237 } }
I20260812 17:52:44.157481 13813 leader_election.cc:302] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fc2bbcc98b54e33a3469c60c9633348; no voters:
I20260812 17:52:44.157862 13813 leader_election.cc:288] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:44.158088 13832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:44.158406 13832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [term 1 LEADER]: Becoming Leader. State: Replica: 2fc2bbcc98b54e33a3469c60c9633348, State: Running, Role: LEADER
I20260812 17:52:44.158756 13832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc2bbcc98b54e33a3469c60c9633348" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36237 } }
I20260812 17:52:44.159291 13813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:44.160650 13834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fc2bbcc98b54e33a3469c60c9633348. Latest consensus state: current_term: 1 leader_uuid: "2fc2bbcc98b54e33a3469c60c9633348" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc2bbcc98b54e33a3469c60c9633348" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36237 } } }
I20260812 17:52:44.160799 13834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:44.161605 13833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fc2bbcc98b54e33a3469c60c9633348" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc2bbcc98b54e33a3469c60c9633348" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36237 } } }
I20260812 17:52:44.161958 13833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348 [sys.catalog]: This master's current role is: LEADER
W20260812 17:52:44.164830 13852 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:52:44.164955 13852 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:52:44.165112 13841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:44.165908 13841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:44.172139 13841 catalog_manager.cc:1347] Generated new cluster ID: 9809302318374d8dbe7cddc3cdaba3fa
I20260812 17:52:44.172245 13841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:44.182319 13841 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:44.182760 13841 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:44.192674 13841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fc2bbcc98b54e33a3469c60c9633348: Generated new TSK 0
I20260812 17:52:44.193687 13841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:44.202907 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 13701
2026-08-12T17:52:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1686 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:44 dist-test-slave-sb2z krb5kdc[13860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:44 dist-test-slave-sb2z krb5kdc[13860](info): set up 2 sockets
Aug 12 17:52:44 dist-test-slave-sb2z krb5kdc[13860](info): commencing operation
krb5kdc: starting...
W20260812 17:52:45.478142 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:45 dist-test-slave-sb2z krb5kdc[13860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 17:52:45.624934 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:45.752939 13876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:45.753334 13876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:45.753427 13876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:45.757798 13876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:45.757920 13876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:45.757963 13876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:45.758026 13876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:45.758083 13876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:45.763952 13876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:45.765184 13876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:45.766300 13876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:45.775811 13885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:45.775795 13882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:45.775795 13889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:45.775900 13876 server_base.cc:1034] running on GCE node
I20260812 17:52:45.776724 13876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:45.777987 13876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:45.779237 13876 hybrid_clock.cc:648] HybridClock initialized: now 1786557165779224 us; error 38 us; skew 500 ppm
Aug 12 17:52:45 dist-test-slave-sb2z krb5kdc[13860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557165, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:45.782558 13876 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 17:52:45.783699 13876 webserver.cc:460] Webserver started at http://127.0.0.1:33205/ using document root <none> and password file <none>
I20260812 17:52:45.784327 13876 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:45.784435 13876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:45.784693 13876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:45.786947 13876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "357b6a41b4ea4cff9fb2ac487facab41"
format_stamp: "Formatted at 2026-08-12 17:52:45 on dist-test-slave-sb2z"
server_key: "3e75ec7d92d01ad16bb5837dfbc0c3a8"
server_key_iv: "72bdb91b73c470c94aacc4e42e51b7ed"
server_key_version: "encryptionkey@0"
I20260812 17:52:45.787659 13876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "357b6a41b4ea4cff9fb2ac487facab41"
format_stamp: "Formatted at 2026-08-12 17:52:45 on dist-test-slave-sb2z"
server_key: "3e75ec7d92d01ad16bb5837dfbc0c3a8"
server_key_iv: "72bdb91b73c470c94aacc4e42e51b7ed"
server_key_version: "encryptionkey@0"
I20260812 17:52:45.792114 13876 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:52:45.795149 13901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:45.796717 13876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:45.796904 13876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "357b6a41b4ea4cff9fb2ac487facab41"
format_stamp: "Formatted at 2026-08-12 17:52:45 on dist-test-slave-sb2z"
server_key: "3e75ec7d92d01ad16bb5837dfbc0c3a8"
server_key_iv: "72bdb91b73c470c94aacc4e42e51b7ed"
server_key_version: "encryptionkey@0"
I20260812 17:52:45.797079 13876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:45.817909 13876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:45.821692 13876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:45.821944 13876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:45.830405 13876 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35693
I20260812 17:52:45.830420 13983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35693 every 8 connection(s)
I20260812 17:52:45.831604 13876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:45.834112 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 13876
I20260812 17:52:45.834280 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:45.834578 13464 external_mini_cluster.cc:1428] Setting key 145fc657b8fa30fb419fa957d1eae982
I20260812 17:52:45.835000 13985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:45.842948 13985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41: Bootstrap starting.
I20260812 17:52:45.845824 13985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41: Neither blocks nor log segments found. Creating new log.
Aug 12 17:52:45 dist-test-slave-sb2z krb5kdc[13860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 17:52:45.846830 13985 log.cc:824] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:45.849874 13985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41: No bootstrap required, opened a new log
I20260812 17:52:45.853317 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:45.836285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:45913 (local address 127.0.0.1:35693)
0812 17:52:45.836763 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:45.836792 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:45.836839 (+ 47us) server_negotiation.cc:407] Connection header received
0812 17:52:45.837958 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:45.837988 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:45.838487 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:45.838925 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:45.839914 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:45.841436 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:45.842474 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:45.843188 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:45.843359 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:45.846698 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:45.846704 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:45.846720 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:45.846753 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:45.849159 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:45.849877 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:45.849886 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:45.849896 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:45.850072 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:45.850522 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:45.850527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:45.850529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:45.851133 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:45.851378 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:45.851788 (+ 410us) server_negotiation.cc:299] Negotiation successful
0812 17:52:45.852065 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":141,"threads_started":1}
I20260812 17:52:45.855319 13985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357b6a41b4ea4cff9fb2ac487facab41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35693 } }
I20260812 17:52:45.855690 13985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:45.855834 13985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 357b6a41b4ea4cff9fb2ac487facab41, State: Initialized, Role: FOLLOWER
I20260812 17:52:45.856418 13985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357b6a41b4ea4cff9fb2ac487facab41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35693 } }
I20260812 17:52:45.856572 13985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:45.856694 13985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:45.856845 13985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:45.856999 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:45.858022 13985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357b6a41b4ea4cff9fb2ac487facab41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35693 } }
I20260812 17:52:45.858454 13985 leader_election.cc:302] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 357b6a41b4ea4cff9fb2ac487facab41; no voters:
I20260812 17:52:45.858879 13985 leader_election.cc:288] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:45.858978 13995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:45.859236 13995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [term 1 LEADER]: Becoming Leader. State: Replica: 357b6a41b4ea4cff9fb2ac487facab41, State: Running, Role: LEADER
I20260812 17:52:45.859648 13995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357b6a41b4ea4cff9fb2ac487facab41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35693 } }
I20260812 17:52:45.860375 13985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:45.861373 13997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 357b6a41b4ea4cff9fb2ac487facab41. Latest consensus state: current_term: 1 leader_uuid: "357b6a41b4ea4cff9fb2ac487facab41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357b6a41b4ea4cff9fb2ac487facab41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35693 } } }
I20260812 17:52:45.861554 13997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:45.861932 13996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "357b6a41b4ea4cff9fb2ac487facab41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357b6a41b4ea4cff9fb2ac487facab41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35693 } } }
I20260812 17:52:45.862025 13996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:45.862738 14013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:45.866122 14013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:45.873219 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:45.859183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:36521 (local address 127.0.0.1:35693)
0812 17:52:45.859304 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:45.859310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:45.863522 (+ 4212us) server_negotiation.cc:407] Connection header received
0812 17:52:45.864122 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:45.864126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:45.864188 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:45.864293 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:45.865118 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:45.865701 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:45.866730 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:45.867002 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:45.867135 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:45.869127 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:45.869134 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:45.869137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:45.869180 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:45.871082 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:45.871805 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:45.871808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:45.871810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:45.871903 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:45.872280 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:45.872288 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:45.872289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:45.872474 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:45.872648 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:45.872931 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 17:52:45.873038 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 17:52:45.877159 14013 catalog_manager.cc:1347] Generated new cluster ID: eca4558d8efc49aeb0c0eb9a23c2c69f
I20260812 17:52:45.877252 14013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:45.908159 14013 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:45.909552 14013 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:45.916486 14013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 357b6a41b4ea4cff9fb2ac487facab41: Generated new TSK 0
I20260812 17:52:45.917279 14013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:45.928280 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 13876
W20260812 17:52:45.947993 13856 connection.cc:535] client connection to 127.0.0.1:35693 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35693): Connection reset by peer (error 104)
2026-08-12T17:52:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1729 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:46 dist-test-slave-sb2z krb5kdc[14026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:46 dist-test-slave-sb2z krb5kdc[14026](info): set up 2 sockets
Aug 12 17:52:46 dist-test-slave-sb2z krb5kdc[14026](info): commencing operation
krb5kdc: starting...
W20260812 17:52:47.224515 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:47 dist-test-slave-sb2z krb5kdc[14026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 17:52:47.372126 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:47.495337 14047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:47.495759 14047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:47.495850 14047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:47.500007 14047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:47.500123 14047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:47.500162 14047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:47.500201 14047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:47.500236 14047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:47.505889 14047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:47.507098 14047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:47.508208 14047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:47.518159 14053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:47.518159 14058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:47.518163 14055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:47.518254 14047 server_base.cc:1034] running on GCE node
I20260812 17:52:47.519198 14047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:47.520557 14047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:47.521767 14047 hybrid_clock.cc:648] HybridClock initialized: now 1786557167521743 us; error 54 us; skew 500 ppm
Aug 12 17:52:47 dist-test-slave-sb2z krb5kdc[14026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557167, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:47.524942 14047 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 17:52:47.526077 14047 webserver.cc:460] Webserver started at http://127.0.0.1:34551/ using document root <none> and password file <none>
I20260812 17:52:47.526742 14047 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:47.526850 14047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:47.527114 14047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:47.529551 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "95470cc2cd284407938cc40daa803088"
format_stamp: "Formatted at 2026-08-12 17:52:47 on dist-test-slave-sb2z"
server_key: "dadc7f0cc8519f573cbf76713c386495"
server_key_iv: "9d94c716df450eab54e0343736305377"
server_key_version: "encryptionkey@0"
I20260812 17:52:47.530205 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "95470cc2cd284407938cc40daa803088"
format_stamp: "Formatted at 2026-08-12 17:52:47 on dist-test-slave-sb2z"
server_key: "dadc7f0cc8519f573cbf76713c386495"
server_key_iv: "9d94c716df450eab54e0343736305377"
server_key_version: "encryptionkey@0"
I20260812 17:52:47.534689 14047 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:52:47.537744 14068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:47.539335 14047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 17:52:47.539602 14047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "95470cc2cd284407938cc40daa803088"
format_stamp: "Formatted at 2026-08-12 17:52:47 on dist-test-slave-sb2z"
server_key: "dadc7f0cc8519f573cbf76713c386495"
server_key_iv: "9d94c716df450eab54e0343736305377"
server_key_version: "encryptionkey@0"
I20260812 17:52:47.539777 14047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:47.559829 14047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:47.563367 14047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:47.563637 14047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:47.571656 14047 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36195
I20260812 17:52:47.571686 14153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36195 every 8 connection(s)
I20260812 17:52:47.572682 14047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:47.575489 14154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:47.581307 14154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088: Bootstrap starting.
I20260812 17:52:47.581686 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 14047
I20260812 17:52:47.581842 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:47.582166 13464 external_mini_cluster.cc:1428] Setting key f0f65526e27bb57d16955c5b16124ebf
I20260812 17:52:47.584240 14154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:47.585013 14154 log.cc:824] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:47.586977 14154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088: No bootstrap required, opened a new log
I20260812 17:52:47.590544 14154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95470cc2cd284407938cc40daa803088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36195 } }
I20260812 17:52:47.590821 14154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:47.590903 14154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95470cc2cd284407938cc40daa803088, State: Initialized, Role: FOLLOWER
I20260812 17:52:47.591428 14154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95470cc2cd284407938cc40daa803088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36195 } }
I20260812 17:52:47.591652 14154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:47.591747 14154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:47.591854 14154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:47.592588 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:47.583684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:53255 (local address 127.0.0.1:36195)
0812 17:52:47.584208 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:47.584225 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:47.584266 (+ 41us) server_negotiation.cc:407] Connection header received
0812 17:52:47.585067 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:47.585085 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:47.585508 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:47.585874 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:47.586337 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:47.587473 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:47.588495 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:47.589028 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:47.589227 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:47.590250 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:47.590255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:47.590270 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:47.590704 (+ 434us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:47.590763 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:47.590788 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:47.590968 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:47.591190 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 17:52:47.591509 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":134,"threads_started":1}
I20260812 17:52:47.592945 14154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95470cc2cd284407938cc40daa803088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36195 } }
I20260812 17:52:47.593328 14154 leader_election.cc:302] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95470cc2cd284407938cc40daa803088; no voters:
I20260812 17:52:47.593690 14154 leader_election.cc:288] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:47.593813 14161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:47.594065 14161 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [term 1 LEADER]: Becoming Leader. State: Replica: 95470cc2cd284407938cc40daa803088, State: Running, Role: LEADER
I20260812 17:52:47.594447 14161 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95470cc2cd284407938cc40daa803088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36195 } }
I20260812 17:52:47.595148 14154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:47.596165 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:47.596272 14164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95470cc2cd284407938cc40daa803088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95470cc2cd284407938cc40daa803088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36195 } } }
I20260812 17:52:47.596402 14164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:47.596720 14168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95470cc2cd284407938cc40daa803088. Latest consensus state: current_term: 1 leader_uuid: "95470cc2cd284407938cc40daa803088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95470cc2cd284407938cc40daa803088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36195 } } }
I20260812 17:52:47.596801 14168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:47.597055 14178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:47.601007 14178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:47.604478 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:47.598543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:59007 (local address 127.0.0.1:36195)
0812 17:52:47.598707 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:47.598714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:47.599266 (+ 552us) server_negotiation.cc:407] Connection header received
0812 17:52:47.599375 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:47.599378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:47.599429 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:47.599518 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:47.599867 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:47.600636 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:47.601820 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:47.602176 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:47.602295 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:47.603922 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:47.603931 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:47.603934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:47.603948 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:47.603974 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:47.603985 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:47.604145 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:47.604292 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 17:52:47.604330 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 17:52:47.607395 14178 catalog_manager.cc:1347] Generated new cluster ID: 40b9e2a2e4064316bd923eb619c3c073
I20260812 17:52:47.607472 14178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:47.620522 14178 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:47.621707 14178 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:47.632478 14178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95470cc2cd284407938cc40daa803088: Generated new TSK 0
I20260812 17:52:47.633308 14178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:47.644012 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 14047
W20260812 17:52:47.657590 14024 connection.cc:535] client connection to 127.0.0.1:36195 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36195): Connection reset by peer (error 104)
2026-08-12T17:52:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1709 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:47 dist-test-slave-sb2z krb5kdc[14208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:47 dist-test-slave-sb2z krb5kdc[14208](info): set up 2 sockets
Aug 12 17:52:47 dist-test-slave-sb2z krb5kdc[14208](info): commencing operation
krb5kdc: starting...
W20260812 17:52:48.946216 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:49 dist-test-slave-sb2z krb5kdc[14208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 17:52:49.092028 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42565
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:49.219187 14225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:49.219486 14225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:49.219615 14225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:49.223803 14225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:49.223884 14225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:49.223909 14225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:49.223928 14225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:49.223943 14225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:49.229694 14225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:49.230924 14225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:49.231959 14225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:49.241760 14234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:49.241755 14232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:49.242121 14225 server_base.cc:1034] running on GCE node
W20260812 17:52:49.241763 14240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:49.242892 14225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:49.244190 14225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:49.245395 14225 hybrid_clock.cc:648] HybridClock initialized: now 1786557169245358 us; error 53 us; skew 500 ppm
Aug 12 17:52:49 dist-test-slave-sb2z krb5kdc[14208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557169, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:49.252653 14225 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 17:52:49.253816 14225 webserver.cc:460] Webserver started at http://127.0.0.1:33305/ using document root <none> and password file <none>
I20260812 17:52:49.254459 14225 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:49.254555 14225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:49.254827 14225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:49.257113 14225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "f8e8e509c6154e5f920cf2639f6a632d"
format_stamp: "Formatted at 2026-08-12 17:52:49 on dist-test-slave-sb2z"
server_key: "5e5ed0a2fd38ba191d6d3d46b06ce3f9"
server_key_iv: "463235eb69a4c89c103655f4e01ea87e"
server_key_version: "encryptionkey@0"
I20260812 17:52:49.257742 14225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "f8e8e509c6154e5f920cf2639f6a632d"
format_stamp: "Formatted at 2026-08-12 17:52:49 on dist-test-slave-sb2z"
server_key: "5e5ed0a2fd38ba191d6d3d46b06ce3f9"
server_key_iv: "463235eb69a4c89c103655f4e01ea87e"
server_key_version: "encryptionkey@0"
I20260812 17:52:49.261976 14225 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:52:49.264737 14251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:49.266094 14225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 17:52:49.266261 14225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "f8e8e509c6154e5f920cf2639f6a632d"
format_stamp: "Formatted at 2026-08-12 17:52:49 on dist-test-slave-sb2z"
server_key: "5e5ed0a2fd38ba191d6d3d46b06ce3f9"
server_key_iv: "463235eb69a4c89c103655f4e01ea87e"
server_key_version: "encryptionkey@0"
I20260812 17:52:49.266390 14225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:49.271759 14225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:49.275246 14225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:49.275486 14225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:49.283879 14225 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42565
I20260812 17:52:49.283906 14337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42565 every 8 connection(s)
I20260812 17:52:49.285020 14225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:49.288112 14338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:49.290515 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 14225
I20260812 17:52:49.290683 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:49.290990 13464 external_mini_cluster.cc:1428] Setting key 7474fa88d71290333747176c9a46c9d3
I20260812 17:52:49.294792 14338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d: Bootstrap starting.
I20260812 17:52:49.296846 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:49.292643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:57907 (local address 127.0.0.1:42565)
0812 17:52:49.293156 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:49.293179 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:49.293221 (+ 42us) server_negotiation.cc:407] Connection header received
0812 17:52:49.293929 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:49.293948 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:49.294256 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:49.294687 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:49.295463 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:49.295468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:49.295482 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:49.295499 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:49.295612 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:49.295647 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:49.295820 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:49.295904 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 17:52:49.296091 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":158,"threads_started":1}
I20260812 17:52:49.297827 14338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:49.298607 14338 log.cc:824] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:49.300454 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:49.301172 14338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d: No bootstrap required, opened a new log
I20260812 17:52:49.304555 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:49.302743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:45629 (local address 127.0.0.1:42565)
0812 17:52:49.302886 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:49.302891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:49.303093 (+ 202us) server_negotiation.cc:407] Connection header received
0812 17:52:49.303215 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:49.303217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:49.303250 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:49.303362 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:49.304103 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:49.304106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:49.304108 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:49.304120 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:49.304148 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:49.304159 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:49.304236 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:49.304361 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 17:52:49.304405 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:52:49.305953 14338 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e509c6154e5f920cf2639f6a632d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42565 } }
I20260812 17:52:49.306231 14338 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:49.306295 14338 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e8e509c6154e5f920cf2639f6a632d, State: Initialized, Role: FOLLOWER
I20260812 17:52:49.306722 14338 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e509c6154e5f920cf2639f6a632d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42565 } }
I20260812 17:52:49.306883 14338 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:49.306957 14338 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:49.307070 14338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:49.308142 14338 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e509c6154e5f920cf2639f6a632d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42565 } }
I20260812 17:52:49.308517 14338 leader_election.cc:302] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8e8e509c6154e5f920cf2639f6a632d; no voters:
I20260812 17:52:49.308868 14338 leader_election.cc:288] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:49.309028 14358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:49.309319 14358 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [term 1 LEADER]: Becoming Leader. State: Replica: f8e8e509c6154e5f920cf2639f6a632d, State: Running, Role: LEADER
I20260812 17:52:49.309711 14358 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e509c6154e5f920cf2639f6a632d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42565 } }
I20260812 17:52:49.310261 14338 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:49.311652 14362 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8e8e509c6154e5f920cf2639f6a632d. Latest consensus state: current_term: 1 leader_uuid: "f8e8e509c6154e5f920cf2639f6a632d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e509c6154e5f920cf2639f6a632d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42565 } } }
I20260812 17:52:49.311796 14362 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:49.312150 14361 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8e8e509c6154e5f920cf2639f6a632d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e509c6154e5f920cf2639f6a632d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42565 } } }
I20260812 17:52:49.312253 14375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:49.312260 14361 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:49.315299 14375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:49.324506 14375 catalog_manager.cc:1347] Generated new cluster ID: 3ee2dd6d7b2f4aeab40f5ac890d63295
I20260812 17:52:49.324605 14375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:49.342881 14375 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:49.343272 14375 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:49.354636 14375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8e8e509c6154e5f920cf2639f6a632d: Generated new TSK 0
I20260812 17:52:49.355469 14375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:49.364284 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 14225
2026-08-12T17:52:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1714 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:49 dist-test-slave-sb2z krb5kdc[14395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:49 dist-test-slave-sb2z krb5kdc[14395](info): set up 2 sockets
Aug 12 17:52:49 dist-test-slave-sb2z krb5kdc[14395](info): commencing operation
krb5kdc: starting...
W20260812 17:52:50.624478 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:50 dist-test-slave-sb2z krb5kdc[14395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 17:52:50.779928 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:50.911693 14416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:50.912086 14416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:50.912173 14416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:50.916422 14416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:50.916551 14416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:50.916590 14416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:50.916630 14416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:50.916662 14416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:50.922492 14416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:50.923755 14416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:50.924844 14416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:50.936609 14424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:50.936655 14426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:50.936601 14430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:50.937179 14416 server_base.cc:1034] running on GCE node
I20260812 17:52:50.937839 14416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:50.939098 14416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:50.940315 14416 hybrid_clock.cc:648] HybridClock initialized: now 1786557170940283 us; error 49 us; skew 500 ppm
Aug 12 17:52:50 dist-test-slave-sb2z krb5kdc[14395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557170, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:50.943779 14416 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 17:52:50.944821 14416 webserver.cc:460] Webserver started at http://127.0.0.1:41591/ using document root <none> and password file <none>
I20260812 17:52:50.945526 14416 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:50.945600 14416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:50.945833 14416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:50.948308 14416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "057c50ab7d7244eb8c69cf4d0bc7fa10"
format_stamp: "Formatted at 2026-08-12 17:52:50 on dist-test-slave-sb2z"
server_key: "1159fd43cf2977a3d3dc29798b77985e"
server_key_iv: "dbc2b19aaf2c25c4d44c93c521f12eab"
server_key_version: "encryptionkey@0"
I20260812 17:52:50.948928 14416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "057c50ab7d7244eb8c69cf4d0bc7fa10"
format_stamp: "Formatted at 2026-08-12 17:52:50 on dist-test-slave-sb2z"
server_key: "1159fd43cf2977a3d3dc29798b77985e"
server_key_iv: "dbc2b19aaf2c25c4d44c93c521f12eab"
server_key_version: "encryptionkey@0"
I20260812 17:52:50.953210 14416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:52:50.956046 14446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:50.957777 14416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:50.957973 14416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "057c50ab7d7244eb8c69cf4d0bc7fa10"
format_stamp: "Formatted at 2026-08-12 17:52:50 on dist-test-slave-sb2z"
server_key: "1159fd43cf2977a3d3dc29798b77985e"
server_key_iv: "dbc2b19aaf2c25c4d44c93c521f12eab"
server_key_version: "encryptionkey@0"
I20260812 17:52:50.958120 14416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:50.973989 14416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:50.977662 14416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:50.977916 14416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:50.986392 14416 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46607
I20260812 17:52:50.986516 14531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46607 every 8 connection(s)
I20260812 17:52:50.987684 14416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:50.989058 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 14416
I20260812 17:52:50.989188 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:50.989516 13464 external_mini_cluster.cc:1428] Setting key 3b73d769e5035d89f9f60353a15db274
I20260812 17:52:50.991392 14532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:50.997627 14532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10: Bootstrap starting.
Aug 12 17:52:50 dist-test-slave-sb2z krb5kdc[14395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 17:52:51.000435 14532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:51.001492 14532 log.cc:824] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:51.003778 14532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10: No bootstrap required, opened a new log
I20260812 17:52:51.005687 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:50.991136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:40971 (local address 127.0.0.1:46607)
0812 17:52:50.991762 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:50.991780 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:50.991822 (+ 42us) server_negotiation.cc:407] Connection header received
0812 17:52:50.992574 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:50.992593 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:50.992969 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:50.993399 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:50.994324 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:50.995429 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:50.996428 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:50.996952 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:50.997157 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:51.000332 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:51.000338 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:51.000352 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:51.000381 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:51.002500 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:51.003056 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:51.003061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:51.003066 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:51.003181 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:51.003599 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:51.003602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:51.003603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:51.004112 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:51.004374 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:51.004708 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 17:52:51.004959 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":180,"threads_started":1}
I20260812 17:52:51.008366 14532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057c50ab7d7244eb8c69cf4d0bc7fa10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46607 } }
I20260812 17:52:51.008692 14532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:51.008764 14532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 057c50ab7d7244eb8c69cf4d0bc7fa10, State: Initialized, Role: FOLLOWER
I20260812 17:52:51.009268 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:51.009305 14532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057c50ab7d7244eb8c69cf4d0bc7fa10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46607 } }
I20260812 17:52:51.009526 14532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:51.009624 14532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:51.009744 14532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:51.010928 14532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057c50ab7d7244eb8c69cf4d0bc7fa10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46607 } }
I20260812 17:52:51.011363 14532 leader_election.cc:302] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 057c50ab7d7244eb8c69cf4d0bc7fa10; no voters:
I20260812 17:52:51.012003 14532 leader_election.cc:288] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:51.012076 14546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:51.012337 14546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [term 1 LEADER]: Becoming Leader. State: Replica: 057c50ab7d7244eb8c69cf4d0bc7fa10, State: Running, Role: LEADER
I20260812 17:52:51.012740 14546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057c50ab7d7244eb8c69cf4d0bc7fa10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46607 } }
I20260812 17:52:51.013549 14532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:51.014703 14548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 057c50ab7d7244eb8c69cf4d0bc7fa10. Latest consensus state: current_term: 1 leader_uuid: "057c50ab7d7244eb8c69cf4d0bc7fa10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057c50ab7d7244eb8c69cf4d0bc7fa10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46607 } } }
I20260812 17:52:51.014858 14548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:51.015288 14547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "057c50ab7d7244eb8c69cf4d0bc7fa10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057c50ab7d7244eb8c69cf4d0bc7fa10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46607 } } }
I20260812 17:52:51.015504 14547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:51.016204 14564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:51.019879 14564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:51.022722 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:51.011753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:55083 (local address 127.0.0.1:46607)
0812 17:52:51.011986 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:51.011993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:51.012104 (+ 111us) server_negotiation.cc:407] Connection header received
0812 17:52:51.012278 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:51.012282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:51.012342 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:51.012426 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:51.013444 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:51.014284 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:51.015618 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:51.016001 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:51.016593 (+ 592us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:51.017969 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:51.017973 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:51.017977 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:51.018005 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:51.020097 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:51.021049 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:51.021052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:51.021054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:51.021152 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:51.021621 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:51.021625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:51.021626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:51.021873 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:51.022041 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:51.022423 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 17:52:51.022554 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 17:52:51.026664 14564 catalog_manager.cc:1347] Generated new cluster ID: 0f3418e5f16e49118399cfaf182a4789
I20260812 17:52:51.026747 14564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:51.042366 14564 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:51.043895 14564 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:51.053699 14564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 057c50ab7d7244eb8c69cf4d0bc7fa10: Generated new TSK 0
I20260812 17:52:51.054620 14564 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:51.062543 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 14416
W20260812 17:52:51.080408 14393 connection.cc:535] client connection to 127.0.0.1:46607 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46607): Connection reset by peer (error 104)
2026-08-12T17:52:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1708 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:51 dist-test-slave-sb2z krb5kdc[14591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:51 dist-test-slave-sb2z krb5kdc[14591](info): set up 2 sockets
Aug 12 17:52:51 dist-test-slave-sb2z krb5kdc[14591](info): commencing operation
krb5kdc: starting...
W20260812 17:52:52.403347 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:52 dist-test-slave-sb2z krb5kdc[14591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 17:52:52.548098 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40597
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:52.671906 14615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:52.672272 14615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:52.672353 14615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:52.676541 14615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:52.676659 14615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:52.676702 14615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:52.676743 14615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:52.676783 14615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:52.682418 14615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40597
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:52.683709 14615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:52.684790 14615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:52.694303 14621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:52.694285 14624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:52.694314 14626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:52.694633 14615 server_base.cc:1034] running on GCE node
I20260812 17:52:52.695317 14615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:52.696610 14615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:52.697788 14615 hybrid_clock.cc:648] HybridClock initialized: now 1786557172697767 us; error 34 us; skew 500 ppm
Aug 12 17:52:52 dist-test-slave-sb2z krb5kdc[14591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557172, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:52.701004 14615 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 17:52:52.702071 14615 webserver.cc:460] Webserver started at http://127.0.0.1:38477/ using document root <none> and password file <none>
I20260812 17:52:52.702705 14615 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:52.702775 14615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:52.703006 14615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:52.705242 14615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "2c6e55f9ce9e4b0aba97db6b7cc63d06"
format_stamp: "Formatted at 2026-08-12 17:52:52 on dist-test-slave-sb2z"
server_key: "875f82f4ac1de677b72e8ed23665f116"
server_key_iv: "f96e601fc7433c4988990e7097c035b0"
server_key_version: "encryptionkey@0"
I20260812 17:52:52.705876 14615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "2c6e55f9ce9e4b0aba97db6b7cc63d06"
format_stamp: "Formatted at 2026-08-12 17:52:52 on dist-test-slave-sb2z"
server_key: "875f82f4ac1de677b72e8ed23665f116"
server_key_iv: "f96e601fc7433c4988990e7097c035b0"
server_key_version: "encryptionkey@0"
I20260812 17:52:52.709952 14615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:52:52.712641 14640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:52.714114 14615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:52.714285 14615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "2c6e55f9ce9e4b0aba97db6b7cc63d06"
format_stamp: "Formatted at 2026-08-12 17:52:52 on dist-test-slave-sb2z"
server_key: "875f82f4ac1de677b72e8ed23665f116"
server_key_iv: "f96e601fc7433c4988990e7097c035b0"
server_key_version: "encryptionkey@0"
I20260812 17:52:52.714424 14615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:52.742817 14615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:52.746943 14615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:52.747206 14615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:52.755532 14615 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40597
I20260812 17:52:52.755581 14723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40597 every 8 connection(s)
I20260812 17:52:52.756651 14615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:52.757102 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 14615
I20260812 17:52:52.757269 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:52.757561 13464 external_mini_cluster.cc:1428] Setting key ad75a8de8637cc5d9d04a4f81c4fdb3c
I20260812 17:52:52.759917 14724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:52.765961 14724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06: Bootstrap starting.
I20260812 17:52:52.767334 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:52.759184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:58635 (local address 127.0.0.1:40597)
0812 17:52:52.759673 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:52.759696 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:52.759736 (+ 40us) server_negotiation.cc:407] Connection header received
0812 17:52:52.760463 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:52.760482 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:52.760845 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:52.761227 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:52.761662 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:52.762693 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:52.763691 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:52.764188 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:52.764416 (+ 228us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:52.765482 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:52.765487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:52.765501 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:52.765906 (+ 405us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:52.765965 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:52.765988 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:52.766157 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:52.766396 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 17:52:52.766556 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":162,"threads_started":1}
I20260812 17:52:52.768544 14724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:52.769474 14724 log.cc:824] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:52.770676 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:52.771997 14724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06: No bootstrap required, opened a new log
I20260812 17:52:52.775460 14724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c6e55f9ce9e4b0aba97db6b7cc63d06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40597 } }
I20260812 17:52:52.775825 14724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:52.775949 14724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c6e55f9ce9e4b0aba97db6b7cc63d06, State: Initialized, Role: FOLLOWER
I20260812 17:52:52.776520 14724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c6e55f9ce9e4b0aba97db6b7cc63d06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40597 } }
I20260812 17:52:52.776727 14724 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:52.776809 14724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:52.776947 14724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:52.777704 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:52.772805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:49237 (local address 127.0.0.1:40597)
0812 17:52:52.772929 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:52.772933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:52.773164 (+ 231us) server_negotiation.cc:407] Connection header received
0812 17:52:52.773336 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:52.773339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:52.773384 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:52.773471 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:52.773763 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:52.774663 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:52.775488 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:52.775794 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:52.775918 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:52.777104 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:52.777111 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:52.777119 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:52.777135 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:52.777164 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:52.777176 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:52.777267 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:52.777469 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 17:52:52.777535 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 17:52:52.778374 14724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c6e55f9ce9e4b0aba97db6b7cc63d06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40597 } }
I20260812 17:52:52.778774 14724 leader_election.cc:302] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c6e55f9ce9e4b0aba97db6b7cc63d06; no voters:
I20260812 17:52:52.779258 14724 leader_election.cc:288] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:52.779337 14741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:52.779621 14741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [term 1 LEADER]: Becoming Leader. State: Replica: 2c6e55f9ce9e4b0aba97db6b7cc63d06, State: Running, Role: LEADER
I20260812 17:52:52.779958 14741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c6e55f9ce9e4b0aba97db6b7cc63d06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40597 } }
I20260812 17:52:52.780750 14724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:52.781869 14742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c6e55f9ce9e4b0aba97db6b7cc63d06. Latest consensus state: current_term: 1 leader_uuid: "2c6e55f9ce9e4b0aba97db6b7cc63d06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c6e55f9ce9e4b0aba97db6b7cc63d06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40597 } } }
I20260812 17:52:52.782034 14742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:52.782342 14745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c6e55f9ce9e4b0aba97db6b7cc63d06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c6e55f9ce9e4b0aba97db6b7cc63d06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40597 } } }
I20260812 17:52:52.782445 14745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:52.782578 14755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:52.786013 14755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:52.792403 14755 catalog_manager.cc:1347] Generated new cluster ID: 67016f48c8294672bfafd7a697064c42
I20260812 17:52:52.792501 14755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:52.815181 14755 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:52.816270 14755 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:52.826037 14755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c6e55f9ce9e4b0aba97db6b7cc63d06: Generated new TSK 0
I20260812 17:52:52.827050 14755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:52.840225 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 14615
W20260812 17:52:52.858546 14588 connection.cc:535] client connection to 127.0.0.1:40597 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40597): Connection reset by peer (error 104)
2026-08-12T17:52:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1778 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:52 dist-test-slave-sb2z krb5kdc[14773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:52 dist-test-slave-sb2z krb5kdc[14773](info): set up 2 sockets
Aug 12 17:52:52 dist-test-slave-sb2z krb5kdc[14773](info): commencing operation
krb5kdc: starting...
W20260812 17:52:54.146441 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:54 dist-test-slave-sb2z krb5kdc[14773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 17:52:54.295866 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39975
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:54.423291 14799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:54.423729 14799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:54.423846 14799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:54.428552 14799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:54.428676 14799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:54.428714 14799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:54.428753 14799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:54.428791 14799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:54.434595 14799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39975
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:54.435881 14799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:54.437011 14799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:54.446547 14812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:54.446568 14808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:54.446555 14807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:54.446893 14799 server_base.cc:1034] running on GCE node
I20260812 17:52:54.447893 14799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:54.449159 14799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:54.450373 14799 hybrid_clock.cc:648] HybridClock initialized: now 1786557174450338 us; error 59 us; skew 500 ppm
Aug 12 17:52:54 dist-test-slave-sb2z krb5kdc[14773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557174, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:54.457209 14799 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 17:52:54.458314 14799 webserver.cc:460] Webserver started at http://127.0.0.1:34553/ using document root <none> and password file <none>
I20260812 17:52:54.458971 14799 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:54.459077 14799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:54.459347 14799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:54.461707 14799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "bdde490bb8d6442e93f327b22e73eed6"
format_stamp: "Formatted at 2026-08-12 17:52:54 on dist-test-slave-sb2z"
server_key: "a06f295450f4f2a42484da9b84d38dec"
server_key_iv: "65eeb0fabbc541034f375d5f02872d7a"
server_key_version: "encryptionkey@0"
I20260812 17:52:54.462380 14799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "bdde490bb8d6442e93f327b22e73eed6"
format_stamp: "Formatted at 2026-08-12 17:52:54 on dist-test-slave-sb2z"
server_key: "a06f295450f4f2a42484da9b84d38dec"
server_key_iv: "65eeb0fabbc541034f375d5f02872d7a"
server_key_version: "encryptionkey@0"
I20260812 17:52:54.467509 14799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 17:52:54.471081 14820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:54.472997 14799 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260812 17:52:54.473187 14799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "bdde490bb8d6442e93f327b22e73eed6"
format_stamp: "Formatted at 2026-08-12 17:52:54 on dist-test-slave-sb2z"
server_key: "a06f295450f4f2a42484da9b84d38dec"
server_key_iv: "65eeb0fabbc541034f375d5f02872d7a"
server_key_version: "encryptionkey@0"
I20260812 17:52:54.473340 14799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:54.479276 14799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:54.483311 14799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:54.483582 14799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:54.491891 14799 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39975
I20260812 17:52:54.491971 14906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39975 every 8 connection(s)
I20260812 17:52:54.492986 14799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:54.494855 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 14799
I20260812 17:52:54.495065 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:54.495414 13464 external_mini_cluster.cc:1428] Setting key 8a45037e7aded88e0eaef0b1aef9a7c6
I20260812 17:52:54.496826 14907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:54.502318 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:54.497113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:50637 (local address 127.0.0.1:39975)
0812 17:52:54.497682 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:54.497705 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:54.497756 (+ 51us) server_negotiation.cc:407] Connection header received
0812 17:52:54.498741 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:54.498770 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:54.499199 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:54.499711 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:54.500572 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:54.500579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:54.500597 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:54.500611 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:54.500684 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:54.500715 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:54.500921 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:54.501053 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 17:52:54.501245 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":147,"threads_started":1}
I20260812 17:52:54.505474 14907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6: Bootstrap starting.
I20260812 17:52:54.505878 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:54.508452 14907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:54.509620 14907 log.cc:824] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:54.510689 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:54.508566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:43913 (local address 127.0.0.1:39975)
0812 17:52:54.508696 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:54.508700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:54.508993 (+ 293us) server_negotiation.cc:407] Connection header received
0812 17:52:54.509120 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:54.509123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:54.509196 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:54.509278 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:54.510217 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:54.510221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:54.510223 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:54.510238 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:54.510270 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:54.510281 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:54.510379 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:54.510473 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 17:52:54.510548 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 17:52:54.512348 14907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6: No bootstrap required, opened a new log
I20260812 17:52:54.516597 14907 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdde490bb8d6442e93f327b22e73eed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39975 } }
I20260812 17:52:54.516963 14907 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:54.517058 14907 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdde490bb8d6442e93f327b22e73eed6, State: Initialized, Role: FOLLOWER
I20260812 17:52:54.517612 14907 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdde490bb8d6442e93f327b22e73eed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39975 } }
I20260812 17:52:54.517763 14907 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:54.517815 14907 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:54.517900 14907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:54.518959 14907 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdde490bb8d6442e93f327b22e73eed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39975 } }
I20260812 17:52:54.519322 14907 leader_election.cc:302] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdde490bb8d6442e93f327b22e73eed6; no voters:
I20260812 17:52:54.519834 14907 leader_election.cc:288] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:54.519984 14934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:54.520404 14934 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [term 1 LEADER]: Becoming Leader. State: Replica: bdde490bb8d6442e93f327b22e73eed6, State: Running, Role: LEADER
I20260812 17:52:54.520804 14934 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdde490bb8d6442e93f327b22e73eed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39975 } }
I20260812 17:52:54.521502 14907 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:54.522814 14935 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdde490bb8d6442e93f327b22e73eed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdde490bb8d6442e93f327b22e73eed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39975 } } }
I20260812 17:52:54.522985 14935 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:54.523419 14936 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdde490bb8d6442e93f327b22e73eed6. Latest consensus state: current_term: 1 leader_uuid: "bdde490bb8d6442e93f327b22e73eed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdde490bb8d6442e93f327b22e73eed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39975 } } }
I20260812 17:52:54.524286 14936 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:54.524425 14944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:54.527469 14944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:54.534192 14944 catalog_manager.cc:1347] Generated new cluster ID: 35210143ec8b4461926ab5439be689d3
I20260812 17:52:54.534304 14944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:54.553856 14944 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:54.554304 14944 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:54.565905 14944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdde490bb8d6442e93f327b22e73eed6: Generated new TSK 0
I20260812 17:52:54.566926 14944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:54.571118 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 14799
2026-08-12T17:52:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1731 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:54 dist-test-slave-sb2z krb5kdc[14964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:54 dist-test-slave-sb2z krb5kdc[14964](info): set up 2 sockets
Aug 12 17:52:54 dist-test-slave-sb2z krb5kdc[14964](info): commencing operation
krb5kdc: starting...
W20260812 17:52:55.858125 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:55 dist-test-slave-sb2z krb5kdc[14964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:52:56.008562 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:40831
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:34221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:40831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:56.130707 14980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:56.131086 14980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:56.131178 14980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:56.135356 14980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:56.135476 14980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:56.135516 14980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:56.135573 14980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:56.135633 14980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:56.141455 14980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:34221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:40831
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:40831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:56.142727 14980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:56.143863 14980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:56.154295 14990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:56.154246 14987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:56.154467 14980 server_base.cc:1034] running on GCE node
W20260812 17:52:56.154274 14988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:56.155314 14980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:56.156548 14980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:56.157735 14980 hybrid_clock.cc:648] HybridClock initialized: now 1786557176157694 us; error 61 us; skew 500 ppm
Aug 12 17:52:56 dist-test-slave-sb2z krb5kdc[14964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557176, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:56.160863 14980 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:52:56.161824 14980 webserver.cc:460] Webserver started at http://127.13.38.62:44795/ using document root <none> and password file <none>
I20260812 17:52:56.162386 14980 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:56.162448 14980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:56.162636 14980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:56.164847 14980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "c9e8bd236ce34f4ea085554fbc6926fc"
format_stamp: "Formatted at 2026-08-12 17:52:56 on dist-test-slave-sb2z"
server_key: "4e65d8d96cea8fd8f3a9ca85003b623a"
server_key_iv: "e5b435fcb8c3d3f0c6e9b32d3c77e463"
server_key_version: "encryptionkey@0"
I20260812 17:52:56.165408 14980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "c9e8bd236ce34f4ea085554fbc6926fc"
format_stamp: "Formatted at 2026-08-12 17:52:56 on dist-test-slave-sb2z"
server_key: "4e65d8d96cea8fd8f3a9ca85003b623a"
server_key_iv: "e5b435fcb8c3d3f0c6e9b32d3c77e463"
server_key_version: "encryptionkey@0"
I20260812 17:52:56.169382 14980 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 17:52:56.173439 15003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:56.175048 14980 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:52:56.175242 14980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "c9e8bd236ce34f4ea085554fbc6926fc"
format_stamp: "Formatted at 2026-08-12 17:52:56 on dist-test-slave-sb2z"
server_key: "4e65d8d96cea8fd8f3a9ca85003b623a"
server_key_iv: "e5b435fcb8c3d3f0c6e9b32d3c77e463"
server_key_version: "encryptionkey@0"
I20260812 17:52:56.175397 14980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:56.195076 14980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:56.198472 14980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:56.198719 14980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:56.206736 14980 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:40831
I20260812 17:52:56.206812 15078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:40831 every 8 connection(s)
I20260812 17:52:56.207856 14980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:56.211108 15079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:56.217489 15079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc: Bootstrap starting.
I20260812 17:52:56.217895 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 14980
I20260812 17:52:56.218073 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:56.218374 13464 external_mini_cluster.cc:1428] Setting key 644ff2f346c0a5f2d983e0af2a114810
I20260812 17:52:56.220583 15079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:56.221453 15079 log.cc:824] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:56.223723 15079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc: No bootstrap required, opened a new log
Aug 12 17:52:56 dist-test-slave-sb2z krb5kdc[14964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:52:56.228068 15079 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e8bd236ce34f4ea085554fbc6926fc" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40831 } }
I20260812 17:52:56.228380 15079 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:56.228469 15079 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9e8bd236ce34f4ea085554fbc6926fc, State: Initialized, Role: FOLLOWER
I20260812 17:52:56.229077 15079 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e8bd236ce34f4ea085554fbc6926fc" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40831 } }
I20260812 17:52:56.229318 15079 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:56.229413 15079 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:56.229578 15079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:56.230705 15079 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e8bd236ce34f4ea085554fbc6926fc" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40831 } }
I20260812 17:52:56.231135 15079 leader_election.cc:302] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9e8bd236ce34f4ea085554fbc6926fc; no voters:
I20260812 17:52:56.231580 15079 leader_election.cc:288] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:56.231678 15086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:56.231884 15086 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [term 1 LEADER]: Becoming Leader. State: Replica: c9e8bd236ce34f4ea085554fbc6926fc, State: Running, Role: LEADER
I20260812 17:52:56.232292 15086 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e8bd236ce34f4ea085554fbc6926fc" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40831 } }
I20260812 17:52:56.233143 15079 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:56.234154 15089 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9e8bd236ce34f4ea085554fbc6926fc. Latest consensus state: current_term: 1 leader_uuid: "c9e8bd236ce34f4ea085554fbc6926fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e8bd236ce34f4ea085554fbc6926fc" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40831 } } }
I20260812 17:52:56.234544 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:56.219922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39037 (local address 127.13.38.62:40831)
0812 17:52:56.220408 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:56.220424 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:56.220462 (+ 38us) server_negotiation.cc:407] Connection header received
0812 17:52:56.221221 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:56.221241 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:56.221631 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:56.222000 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:56.222858 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:56.223975 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:56.224878 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:56.225378 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:56.225558 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:56.228506 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:56.228513 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:56.228555 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:56.228598 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:56.230646 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:56.231246 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:56.231252 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:56.231259 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:56.231416 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:56.231898 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:56.231902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:56.231904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:56.232496 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:56.232776 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:56.233112 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 17:52:56.233375 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":149,"threads_started":1}
I20260812 17:52:56.234586 15089 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:56.234460 15088 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9e8bd236ce34f4ea085554fbc6926fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e8bd236ce34f4ea085554fbc6926fc" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 40831 } } }
I20260812 17:52:56.235213 15088 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:56.238583 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 17:52:56.239607 15107 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:52:56.239710 15107 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:52:56.239923 15101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:56.241055 15101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:56.247910 15101 catalog_manager.cc:1347] Generated new cluster ID: 31106d7b767d4b3484a9fdccbd8f0bf4
I20260812 17:52:56.248011 15101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:56.254395 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:56.240782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41151 (local address 127.13.38.62:40831)
0812 17:52:56.240922 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:56.240926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:56.243727 (+ 2801us) server_negotiation.cc:407] Connection header received
0812 17:52:56.243908 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:56.243911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:56.243970 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:56.244069 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:56.245123 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:56.246202 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:56.247600 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:56.248056 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:56.248151 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:56.250162 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:56.250167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:56.250171 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:52:56.250205 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:56.252219 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:56.252838 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:56.252842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:56.252844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:56.252954 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:52:56.253331 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:52:56.253335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:52:56.253337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:52:56.253588 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:56.253751 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:56.254085 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 17:52:56.254206 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 17:52:56.260259 15101 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:56.261765 15101 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:56.273242 15101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9e8bd236ce34f4ea085554fbc6926fc: Generated new TSK 0
I20260812 17:52:56.274189 15101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:56.286306 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 14980
W20260812 17:52:56.305681 14960 connection.cc:535] client connection to 127.13.38.62:40831 recv error: Network error: failed to read from TLS socket (remote: 127.13.38.62:40831): Connection reset by peer (error 104)
2026-08-12T17:52:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1714 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:56 dist-test-slave-sb2z krb5kdc[15123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:56 dist-test-slave-sb2z krb5kdc[15123](info): set up 2 sockets
Aug 12 17:52:56 dist-test-slave-sb2z krb5kdc[15123](info): commencing operation
krb5kdc: starting...
W20260812 17:52:57.624984 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:57 dist-test-slave-sb2z krb5kdc[15123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:52:57.768399 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:36939
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:42913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:36939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:57.888659 15150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:57.888955 15150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:57.888999 15150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:57.893152 15150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:57.893224 15150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:57.893244 15150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:57.893261 15150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:57.893276 15150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:57.898818 15150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:42913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:36939
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:36939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:57.900023 15150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:57.901105 15150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:57.910159 15161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:57.910163 15157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:57.910214 15163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:57.910506 15150 server_base.cc:1034] running on GCE node
I20260812 17:52:57.911139 15150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:57.912386 15150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:57.913563 15150 hybrid_clock.cc:648] HybridClock initialized: now 1786557177913528 us; error 51 us; skew 500 ppm
Aug 12 17:52:57 dist-test-slave-sb2z krb5kdc[15123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557177, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:57.916608 15150 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:52:57.917573 15150 webserver.cc:460] Webserver started at http://127.13.38.62:45943/ using document root <none> and password file <none>
I20260812 17:52:57.918190 15150 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:57.918290 15150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:57.918536 15150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:57.920701 15150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "575b31a33889439fa353982ef9a35959"
format_stamp: "Formatted at 2026-08-12 17:52:57 on dist-test-slave-sb2z"
server_key: "08a2ac30bef9c7a7054a9305e89e73ca"
server_key_iv: "ce3e30f72bb33eb6eb75bbaa20a25e71"
server_key_version: "encryptionkey@0"
I20260812 17:52:57.921316 15150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "575b31a33889439fa353982ef9a35959"
format_stamp: "Formatted at 2026-08-12 17:52:57 on dist-test-slave-sb2z"
server_key: "08a2ac30bef9c7a7054a9305e89e73ca"
server_key_iv: "ce3e30f72bb33eb6eb75bbaa20a25e71"
server_key_version: "encryptionkey@0"
I20260812 17:52:57.925434 15150 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 17:52:57.928280 15172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:57.929641 15150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:57.929809 15150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "575b31a33889439fa353982ef9a35959"
format_stamp: "Formatted at 2026-08-12 17:52:57 on dist-test-slave-sb2z"
server_key: "08a2ac30bef9c7a7054a9305e89e73ca"
server_key_iv: "ce3e30f72bb33eb6eb75bbaa20a25e71"
server_key_version: "encryptionkey@0"
I20260812 17:52:57.929947 15150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:57.953171 15150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:57.956913 15150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:57.957149 15150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:57.965467 15150 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:36939
I20260812 17:52:57.965502 15261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:36939 every 8 connection(s)
I20260812 17:52:57.966526 15150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:57.967015 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 15150
I20260812 17:52:57.967185 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:57.967567 13464 external_mini_cluster.cc:1428] Setting key 2288861a94d3ed8d2f60b92fc2b459e0
I20260812 17:52:57.970398 15262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:57.978466 15262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959: Bootstrap starting.
I20260812 17:52:57.979091 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:57.969106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57229 (local address 127.13.38.62:36939)
0812 17:52:57.969583 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:57.969605 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:57.969653 (+ 48us) server_negotiation.cc:407] Connection header received
0812 17:52:57.970645 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:57.970671 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:57.971142 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:57.971602 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:57.972067 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:57.973744 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:57.974751 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:57.975504 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:57.975721 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:57.976776 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:57.976783 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:57.976804 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:57.977371 (+ 567us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:57.977452 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:57.977486 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:57.977654 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:57.977892 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 17:52:57.978042 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":125,"threads_started":1}
I20260812 17:52:57.981487 15262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:57.982681 15262 log.cc:824] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:57.983167 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:57.985268 15262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959: No bootstrap required, opened a new log
I20260812 17:52:57.990459 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:57.985566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36005 (local address 127.13.38.62:36939)
0812 17:52:57.985711 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:57.985716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:57.985907 (+ 191us) server_negotiation.cc:407] Connection header received
0812 17:52:57.986058 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:57.986060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:57.986101 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:57.986172 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:57.986529 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:57.987103 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:57.988386 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:52:57.988646 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:52:57.988741 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:52:57.989764 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:57.989767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:57.989769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:57.989782 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:57.989811 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:57.989822 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:57.989903 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:57.990134 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 17:52:57.990248 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 17:52:57.990401 15262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575b31a33889439fa353982ef9a35959" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 36939 } }
I20260812 17:52:57.990767 15262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:57.990839 15262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575b31a33889439fa353982ef9a35959, State: Initialized, Role: FOLLOWER
I20260812 17:52:57.991334 15262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575b31a33889439fa353982ef9a35959" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 36939 } }
I20260812 17:52:57.991473 15262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:57.991626 15262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:57.991762 15262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:57.992903 15262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575b31a33889439fa353982ef9a35959" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 36939 } }
I20260812 17:52:57.993309 15262 leader_election.cc:302] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 575b31a33889439fa353982ef9a35959; no voters:
I20260812 17:52:57.993834 15262 leader_election.cc:288] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:57.994020 15283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:57.994397 15283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [term 1 LEADER]: Becoming Leader. State: Replica: 575b31a33889439fa353982ef9a35959, State: Running, Role: LEADER
I20260812 17:52:57.994764 15283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575b31a33889439fa353982ef9a35959" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 36939 } }
I20260812 17:52:57.995385 15262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:57.996745 15286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 575b31a33889439fa353982ef9a35959. Latest consensus state: current_term: 1 leader_uuid: "575b31a33889439fa353982ef9a35959" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575b31a33889439fa353982ef9a35959" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 36939 } } }
I20260812 17:52:57.996934 15286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:57.996745 15284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "575b31a33889439fa353982ef9a35959" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575b31a33889439fa353982ef9a35959" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 36939 } } }
I20260812 17:52:57.996999 15284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959 [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:57.997418 15296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:58.000345 15296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:58.007088 15296 catalog_manager.cc:1347] Generated new cluster ID: d0c5306fbbb44fc49673489a41370d63
I20260812 17:52:58.007186 15296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:58.017999 15296 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:58.019057 15296 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:58.026126 15296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 575b31a33889439fa353982ef9a35959: Generated new TSK 0
I20260812 17:52:58.026911 15296 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:58.039177 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 15150
W20260812 17:52:58.050974 15118 connection.cc:535] client connection to 127.13.38.62:36939 recv error: Network error: failed to read from TLS socket (remote: 127.13.38.62:36939): Connection reset by peer (error 104)
2026-08-12T17:52:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1745 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:58 dist-test-slave-sb2z krb5kdc[15317](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:58 dist-test-slave-sb2z krb5kdc[15317](info): set up 2 sockets
Aug 12 17:52:58 dist-test-slave-sb2z krb5kdc[15317](info): commencing operation
krb5kdc: starting...
W20260812 17:52:59.355099 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:52:59 dist-test-slave-sb2z krb5kdc[15317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:52:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:52:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:52:59.500553 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:42673
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:46581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:42673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:52:59.623971 15337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:52:59.624287 15337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:52:59.624370 15337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:52:59.628422 15337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:52:59.628518 15337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:52:59.628556 15337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:52:59.628618 15337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:52:59.628656 15337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:52:59.633994 15337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:46581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:42673
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:42673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:52:59.635149 15337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:52:59.636238 15337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:52:59.644959 15344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:59.645048 15337 server_base.cc:1034] running on GCE node
W20260812 17:52:59.644973 15346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:52:59.645083 15349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:52:59.645799 15337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:52:59.646945 15337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:52:59.648116 15337 hybrid_clock.cc:648] HybridClock initialized: now 1786557179648088 us; error 43 us; skew 500 ppm
Aug 12 17:52:59 dist-test-slave-sb2z krb5kdc[15317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557179, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:52:59.655686 15337 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:52:59.656708 15337 webserver.cc:460] Webserver started at http://127.13.38.62:37691/ using document root <none> and password file <none>
I20260812 17:52:59.657339 15337 fs_manager.cc:359] Metadata directory not provided
I20260812 17:52:59.657442 15337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:52:59.657701 15337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:52:59.660017 15337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "9ce707f8616c4e56a4b79082805bd72a"
format_stamp: "Formatted at 2026-08-12 17:52:59 on dist-test-slave-sb2z"
server_key: "3de94475910ebd8ceb038a714b287962"
server_key_iv: "f9918ecc2329a67cd219d9315f59a7dc"
server_key_version: "encryptionkey@0"
I20260812 17:52:59.660641 15337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "9ce707f8616c4e56a4b79082805bd72a"
format_stamp: "Formatted at 2026-08-12 17:52:59 on dist-test-slave-sb2z"
server_key: "3de94475910ebd8ceb038a714b287962"
server_key_iv: "f9918ecc2329a67cd219d9315f59a7dc"
server_key_version: "encryptionkey@0"
I20260812 17:52:59.665336 15337 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:52:59.668134 15365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:52:59.669507 15337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:52:59.669618 15337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "9ce707f8616c4e56a4b79082805bd72a"
format_stamp: "Formatted at 2026-08-12 17:52:59 on dist-test-slave-sb2z"
server_key: "3de94475910ebd8ceb038a714b287962"
server_key_iv: "f9918ecc2329a67cd219d9315f59a7dc"
server_key_version: "encryptionkey@0"
I20260812 17:52:59.669718 15337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:52:59.674418 15337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:52:59.677901 15337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:52:59.678057 15337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:52:59.685441 15337 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:42673
I20260812 17:52:59.685520 15446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:42673 every 8 connection(s)
I20260812 17:52:59.686468 15337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:52:59.689078 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 15337
I20260812 17:52:59.689244 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:52:59.689563 13464 external_mini_cluster.cc:1428] Setting key 17c36e5fbb2497a6c129a05b61025348
I20260812 17:52:59.689987 15447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:52:59.696372 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:59.691204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33717 (local address 127.13.38.62:42673)
0812 17:52:59.691693 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:59.691713 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:59.691758 (+ 45us) server_negotiation.cc:407] Connection header received
0812 17:52:59.693037 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:59.693068 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:59.693496 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:59.693860 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:59.694928 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:59.694934 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:59.694947 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:59.694961 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:59.695034 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:59.695057 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:59.695193 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:59.695324 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 17:52:59.695482 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":130,"threads_started":1}
I20260812 17:52:59.697849 15447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a: Bootstrap starting.
I20260812 17:52:59.699643 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:52:59.700398 15447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a: Neither blocks nor log segments found. Creating new log.
I20260812 17:52:59.701251 15447 log.cc:824] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a: Log is configured to *not* fsync() on all Append() calls
I20260812 17:52:59.703572 15447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a: No bootstrap required, opened a new log
I20260812 17:52:59.703855 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:52:59.701946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48255 (local address 127.13.38.62:42673)
0812 17:52:59.702092 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 17:52:59.702097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:52:59.702307 (+ 210us) server_negotiation.cc:407] Connection header received
0812 17:52:59.702490 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:52:59.702493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:52:59.702534 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:52:59.702616 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:52:59.703403 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:52:59.703407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:52:59.703410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:52:59.703422 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:52:59.703452 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:52:59.703476 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:52:59.703596 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 17:52:59.703688 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 17:52:59.703729 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 17:52:59.707728 15447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ce707f8616c4e56a4b79082805bd72a" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42673 } }
I20260812 17:52:59.707988 15447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:52:59.708067 15447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ce707f8616c4e56a4b79082805bd72a, State: Initialized, Role: FOLLOWER
I20260812 17:52:59.708526 15447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ce707f8616c4e56a4b79082805bd72a" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42673 } }
I20260812 17:52:59.708678 15447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:52:59.708745 15447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:52:59.708845 15447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:52:59.709849 15447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ce707f8616c4e56a4b79082805bd72a" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42673 } }
I20260812 17:52:59.710294 15447 leader_election.cc:302] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ce707f8616c4e56a4b79082805bd72a; no voters:
I20260812 17:52:59.710822 15447 leader_election.cc:288] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:52:59.710965 15463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:52:59.711270 15463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [term 1 LEADER]: Becoming Leader. State: Replica: 9ce707f8616c4e56a4b79082805bd72a, State: Running, Role: LEADER
I20260812 17:52:59.711685 15463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ce707f8616c4e56a4b79082805bd72a" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42673 } }
I20260812 17:52:59.712410 15447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:52:59.713492 15469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ce707f8616c4e56a4b79082805bd72a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ce707f8616c4e56a4b79082805bd72a" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42673 } } }
I20260812 17:52:59.713637 15469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:59.713929 15470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ce707f8616c4e56a4b79082805bd72a. Latest consensus state: current_term: 1 leader_uuid: "9ce707f8616c4e56a4b79082805bd72a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ce707f8616c4e56a4b79082805bd72a" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42673 } } }
I20260812 17:52:59.714027 15470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a [sys.catalog]: This master's current role is: LEADER
I20260812 17:52:59.714042 15479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:52:59.716952 15479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:52:59.722539 15479 catalog_manager.cc:1347] Generated new cluster ID: d3573bf3a7354067bcf885df4a72b6b4
I20260812 17:52:59.722620 15479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:52:59.743356 15479 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:52:59.743835 15479 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:52:59.751390 15479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ce707f8616c4e56a4b79082805bd72a: Generated new TSK 0
I20260812 17:52:59.752218 15479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:52:59.756593 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 15337
2026-08-12T17:52:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1725 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:52:59 dist-test-slave-sb2z krb5kdc[15504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:52:59 dist-test-slave-sb2z krb5kdc[15504](info): set up 2 sockets
Aug 12 17:52:59 dist-test-slave-sb2z krb5kdc[15504](info): commencing operation
krb5kdc: starting...
W20260812 17:53:01.066332 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:53:01 dist-test-slave-sb2z krb5kdc[15504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:53:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:53:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:53:01.213290 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:44893
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:44893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:53:01.342168 15527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:53:01.342469 15527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:53:01.342515 15527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:53:01.346797 15527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:53:01.346887 15527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:53:01.346908 15527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:53:01.346926 15527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:53:01.346942 15527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:53:01.352794 15527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:44893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:44893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:53:01.354015 15527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:53:01.355186 15527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:53:01.364662 15536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:01.364705 15542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:01.364703 15535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:53:01.365051 15527 server_base.cc:1034] running on GCE node
I20260812 17:53:01.365801 15527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:53:01.367151 15527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:53:01.368383 15527 hybrid_clock.cc:648] HybridClock initialized: now 1786557181368349 us; error 45 us; skew 500 ppm
Aug 12 17:53:01 dist-test-slave-sb2z krb5kdc[15504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557181, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:53:01.371671 15527 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:53:01.372743 15527 webserver.cc:460] Webserver started at http://127.13.38.62:44215/ using document root <none> and password file <none>
I20260812 17:53:01.373377 15527 fs_manager.cc:359] Metadata directory not provided
I20260812 17:53:01.373477 15527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:53:01.373713 15527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:53:01.375934 15527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "8ac6d07d44b249a8ae29b317d0866f40"
format_stamp: "Formatted at 2026-08-12 17:53:01 on dist-test-slave-sb2z"
server_key: "49ef0fbd37e166bc72312f41eeffd985"
server_key_iv: "b4a2287678a578428f28f106f5abd970"
server_key_version: "encryptionkey@0"
I20260812 17:53:01.376521 15527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "8ac6d07d44b249a8ae29b317d0866f40"
format_stamp: "Formatted at 2026-08-12 17:53:01 on dist-test-slave-sb2z"
server_key: "49ef0fbd37e166bc72312f41eeffd985"
server_key_iv: "b4a2287678a578428f28f106f5abd970"
server_key_version: "encryptionkey@0"
I20260812 17:53:01.380517 15527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 17:53:01.383234 15549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:53:01.384629 15527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:53:01.384786 15527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "8ac6d07d44b249a8ae29b317d0866f40"
format_stamp: "Formatted at 2026-08-12 17:53:01 on dist-test-slave-sb2z"
server_key: "49ef0fbd37e166bc72312f41eeffd985"
server_key_iv: "b4a2287678a578428f28f106f5abd970"
server_key_version: "encryptionkey@0"
I20260812 17:53:01.384913 15527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:53:01.407368 15527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:53:01.411302 15527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:53:01.411638 15527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:53:01.419924 15527 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:44893
I20260812 17:53:01.420017 15636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:44893 every 8 connection(s)
I20260812 17:53:01.421053 15527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:53:01.422389 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 15527
I20260812 17:53:01.422591 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:53:01.422926 13464 external_mini_cluster.cc:1428] Setting key 63c525971dcb4c96581b056bc4d5f3af
I20260812 17:53:01.424597 15638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:53:01.431627 15638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40: Bootstrap starting.
Aug 12 17:53:01 dist-test-slave-sb2z krb5kdc[15504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:53:01.434341 15638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40: Neither blocks nor log segments found. Creating new log.
I20260812 17:53:01.435319 15638 log.cc:824] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40: Log is configured to *not* fsync() on all Append() calls
I20260812 17:53:01.437608 15638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40: No bootstrap required, opened a new log
I20260812 17:53:01.439587 15643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:01.424599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43021 (local address 127.13.38.62:44893)
0812 17:53:01.425181 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:01.425201 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:01.425243 (+ 42us) server_negotiation.cc:407] Connection header received
0812 17:53:01.425977 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:01.425996 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:01.426392 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:01.426729 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:01.427765 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:01.428976 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:01.430234 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:01.430771 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:01.430921 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:01.434148 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:01.434154 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:01.434170 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:01.434204 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:01.436441 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:01.436962 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:01.436967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:01.436972 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:01.437095 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:01.437432 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:01.437434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:01.437435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:01.437912 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:01.438136 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:01.438466 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 17:53:01.438778 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":192,"threads_started":1}
I20260812 17:53:01.441551 15638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac6d07d44b249a8ae29b317d0866f40" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44893 } }
I20260812 17:53:01.441821 15638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:53:01.441898 15638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ac6d07d44b249a8ae29b317d0866f40, State: Initialized, Role: FOLLOWER
I20260812 17:53:01.442425 15638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac6d07d44b249a8ae29b317d0866f40" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44893 } }
I20260812 17:53:01.442584 15638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:53:01.442579 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:53:01.442672 15638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:53:01.442776 15638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:53:01.443840 15638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac6d07d44b249a8ae29b317d0866f40" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44893 } }
I20260812 17:53:01.444234 15638 leader_election.cc:302] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ac6d07d44b249a8ae29b317d0866f40; no voters:
I20260812 17:53:01.444590 15638 leader_election.cc:288] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:53:01.444715 15658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:53:01.445081 15658 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [term 1 LEADER]: Becoming Leader. State: Replica: 8ac6d07d44b249a8ae29b317d0866f40, State: Running, Role: LEADER
I20260812 17:53:01.445416 15658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac6d07d44b249a8ae29b317d0866f40" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44893 } }
I20260812 17:53:01.445968 15638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:53:01.446580 15658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ac6d07d44b249a8ae29b317d0866f40. Latest consensus state: current_term: 1 leader_uuid: "8ac6d07d44b249a8ae29b317d0866f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac6d07d44b249a8ae29b317d0866f40" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44893 } } }
I20260812 17:53:01.446780 15658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [sys.catalog]: This master's current role is: LEADER
I20260812 17:53:01.447021 15665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ac6d07d44b249a8ae29b317d0866f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac6d07d44b249a8ae29b317d0866f40" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 44893 } } }
I20260812 17:53:01.447154 15665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40 [sys.catalog]: This master's current role is: LEADER
I20260812 17:53:01.447347 15676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:53:01.450805 15676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:53:01.458022 15676 catalog_manager.cc:1347] Generated new cluster ID: c43654b963ab45ce892e6c108804eeed
I20260812 17:53:01.458105 15676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:53:01.458657 15643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:01.444766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58413 (local address 127.13.38.62:44893)
0812 17:53:01.444888 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:01.444892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:01.449004 (+ 4112us) server_negotiation.cc:407] Connection header received
0812 17:53:01.449180 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:01.449184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:01.449237 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:01.449326 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:01.450311 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:01.451089 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:01.452279 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:01.452683 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:01.452803 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:01.454390 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:01.454394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:01.454397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:01.454434 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:01.456476 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:01.456946 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:01.456950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:01.456952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:01.457047 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:01.457351 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:01.457354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:01.457356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:01.457557 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:01.457713 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:01.458369 (+ 656us) server_negotiation.cc:299] Negotiation successful
0812 17:53:01.458507 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 17:53:01.469187 15676 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:53:01.470707 15676 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:53:01.477514 15676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ac6d07d44b249a8ae29b317d0866f40: Generated new TSK 0
I20260812 17:53:01.478288 15676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:53:01.482803 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 15527
2026-08-12T17:53:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1728 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:53:01 dist-test-slave-sb2z krb5kdc[15691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:53:01 dist-test-slave-sb2z krb5kdc[15691](info): set up 2 sockets
Aug 12 17:53:01 dist-test-slave-sb2z krb5kdc[15691](info): commencing operation
krb5kdc: starting...
W20260812 17:53:02.779448 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:53:02 dist-test-slave-sb2z krb5kdc[15691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:53:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:53:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:53:02.924281 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:33595
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:33595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:53:03.049691 15716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:53:03.050056 15716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:53:03.050148 15716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:53:03.054399 15716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:53:03.054515 15716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:53:03.054550 15716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:53:03.054594 15716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:53:03.054652 15716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:53:03.060384 15716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:33595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:33595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:53:03.061630 15716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:53:03.062706 15716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 17:53:03.071986 15716 server_base.cc:1034] running on GCE node
W20260812 17:53:03.071906 15727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:03.071887 15725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:03.071856 15730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:53:03.072911 15716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:53:03.074180 15716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:53:03.075409 15716 hybrid_clock.cc:648] HybridClock initialized: now 1786557183075393 us; error 31 us; skew 500 ppm
Aug 12 17:53:03 dist-test-slave-sb2z krb5kdc[15691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557183, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:53:03.078497 15716 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:53:03.079474 15716 webserver.cc:460] Webserver started at http://127.13.38.62:39065/ using document root <none> and password file <none>
I20260812 17:53:03.080102 15716 fs_manager.cc:359] Metadata directory not provided
I20260812 17:53:03.080168 15716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:53:03.080344 15716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:53:03.082432 15716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "61d6357dd57c491eb9d73d8c2904f5ff"
format_stamp: "Formatted at 2026-08-12 17:53:03 on dist-test-slave-sb2z"
server_key: "6b892f53330d469f19c3621d45dc0be0"
server_key_iv: "79f118c030565df54aaf1ca60522df39"
server_key_version: "encryptionkey@0"
I20260812 17:53:03.082991 15716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "61d6357dd57c491eb9d73d8c2904f5ff"
format_stamp: "Formatted at 2026-08-12 17:53:03 on dist-test-slave-sb2z"
server_key: "6b892f53330d469f19c3621d45dc0be0"
server_key_iv: "79f118c030565df54aaf1ca60522df39"
server_key_version: "encryptionkey@0"
I20260812 17:53:03.087057 15716 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:53:03.089969 15739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:53:03.091485 15716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 17:53:03.091701 15716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "61d6357dd57c491eb9d73d8c2904f5ff"
format_stamp: "Formatted at 2026-08-12 17:53:03 on dist-test-slave-sb2z"
server_key: "6b892f53330d469f19c3621d45dc0be0"
server_key_iv: "79f118c030565df54aaf1ca60522df39"
server_key_version: "encryptionkey@0"
I20260812 17:53:03.091842 15716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:53:03.101715 15716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:53:03.105199 15716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:53:03.105436 15716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:53:03.113585 15716 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:33595
I20260812 17:53:03.113615 15834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:33595 every 8 connection(s)
I20260812 17:53:03.114609 15716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:53:03.117616 15835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:53:03.123140 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 15716
I20260812 17:53:03.123289 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:53:03.123621 13464 external_mini_cluster.cc:1428] Setting key 41a3057919276cb533e948376ff621ca
I20260812 17:53:03.123731 15835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff: Bootstrap starting.
I20260812 17:53:03.126430 15835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff: Neither blocks nor log segments found. Creating new log.
I20260812 17:53:03.127198 15835 log.cc:824] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff: Log is configured to *not* fsync() on all Append() calls
I20260812 17:53:03.129305 15835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff: No bootstrap required, opened a new log
I20260812 17:53:03.133213 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:03.125164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60537 (local address 127.13.38.62:33595)
0812 17:53:03.125645 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:03.125662 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:03.125699 (+ 37us) server_negotiation.cc:407] Connection header received
0812 17:53:03.126408 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:03.126427 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:03.126852 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:03.127210 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:03.127711 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:03.128686 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:03.129706 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:03.130202 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:03.130393 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:03.131326 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:03.131331 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:03.131347 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:53:03.131831 (+ 484us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:03.131891 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:53:03.131915 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:03.132041 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:03.132199 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 17:53:03.132422 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":150,"threads_started":1}
I20260812 17:53:03.133708 15835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d6357dd57c491eb9d73d8c2904f5ff" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33595 } }
I20260812 17:53:03.134007 15835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:53:03.134058 15835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d6357dd57c491eb9d73d8c2904f5ff, State: Initialized, Role: FOLLOWER
I20260812 17:53:03.134506 15835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d6357dd57c491eb9d73d8c2904f5ff" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33595 } }
I20260812 17:53:03.134634 15835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:53:03.134722 15835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:53:03.134860 15835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:53:03.136125 15835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d6357dd57c491eb9d73d8c2904f5ff" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33595 } }
I20260812 17:53:03.136451 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:53:03.136523 15835 leader_election.cc:302] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61d6357dd57c491eb9d73d8c2904f5ff; no voters:
I20260812 17:53:03.136861 15835 leader_election.cc:288] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:53:03.137032 15841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:53:03.137302 15841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [term 1 LEADER]: Becoming Leader. State: Replica: 61d6357dd57c491eb9d73d8c2904f5ff, State: Running, Role: LEADER
I20260812 17:53:03.137641 15841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d6357dd57c491eb9d73d8c2904f5ff" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33595 } }
I20260812 17:53:03.138304 15835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:53:03.139024 15842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61d6357dd57c491eb9d73d8c2904f5ff. Latest consensus state: current_term: 1 leader_uuid: "61d6357dd57c491eb9d73d8c2904f5ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d6357dd57c491eb9d73d8c2904f5ff" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33595 } } }
I20260812 17:53:03.139183 15842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [sys.catalog]: This master's current role is: LEADER
I20260812 17:53:03.139480 15844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61d6357dd57c491eb9d73d8c2904f5ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d6357dd57c491eb9d73d8c2904f5ff" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33595 } } }
I20260812 17:53:03.139603 15844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff [sys.catalog]: This master's current role is: LEADER
I20260812 17:53:03.139711 15861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:53:03.143083 15861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:53:03.145798 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:03.138773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52403 (local address 127.13.38.62:33595)
0812 17:53:03.138880 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:03.138884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:03.141041 (+ 2157us) server_negotiation.cc:407] Connection header received
0812 17:53:03.141255 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:03.141261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:03.141310 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:03.141363 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:03.141789 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:03.142531 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:03.143659 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:03.143968 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:03.144070 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:03.145114 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:03.145118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:03.145120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:53:03.145135 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:03.145164 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:53:03.145180 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:03.145249 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:03.145622 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 17:53:03.145680 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 17:53:03.149437 15861 catalog_manager.cc:1347] Generated new cluster ID: b79f0ece4e494c6e866add1b18a56518
I20260812 17:53:03.149516 15861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:53:03.165704 15861 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:53:03.166723 15861 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:53:03.172787 15861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61d6357dd57c491eb9d73d8c2904f5ff: Generated new TSK 0
I20260812 17:53:03.173581 15861 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:53:03.178787 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 15716
2026-08-12T17:53:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1689 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:53:03 dist-test-slave-sb2z krb5kdc[15884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:53:03 dist-test-slave-sb2z krb5kdc[15884](info): set up 2 sockets
Aug 12 17:53:03 dist-test-slave-sb2z krb5kdc[15884](info): commencing operation
krb5kdc: starting...
W20260812 17:53:04.458971 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:53:04 dist-test-slave-sb2z krb5kdc[15884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:53:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:53:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:53:04.605129 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:39429
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:39429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:53:04.724890 15905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:53:04.725230 15905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:53:04.725315 15905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:53:04.729599 15905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:53:04.729703 15905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:53:04.729744 15905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:53:04.729784 15905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:53:04.729821 15905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:53:04.735353 15905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:39429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:39429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:53:04.736626 15905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:53:04.737778 15905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:53:04.747380 15913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:04.747412 15911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:53:04.747437 15905 server_base.cc:1034] running on GCE node
W20260812 17:53:04.747380 15918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:53:04.748293 15905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:53:04.749442 15905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:53:04.750663 15905 hybrid_clock.cc:648] HybridClock initialized: now 1786557184750629 us; error 49 us; skew 500 ppm
Aug 12 17:53:04 dist-test-slave-sb2z krb5kdc[15884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557184, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:53:04.757534 15905 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:53:04.758606 15905 webserver.cc:460] Webserver started at http://127.13.38.62:43555/ using document root <none> and password file <none>
I20260812 17:53:04.759258 15905 fs_manager.cc:359] Metadata directory not provided
I20260812 17:53:04.759364 15905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:53:04.759711 15905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:53:04.762413 15905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "9d0fa65e13e74406ba593485c46d7b4b"
format_stamp: "Formatted at 2026-08-12 17:53:04 on dist-test-slave-sb2z"
server_key: "677edc11da7d9e7a655c0208524d853e"
server_key_iv: "2a443af0d9e7172b146f37a68347a35e"
server_key_version: "encryptionkey@0"
I20260812 17:53:04.763101 15905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "9d0fa65e13e74406ba593485c46d7b4b"
format_stamp: "Formatted at 2026-08-12 17:53:04 on dist-test-slave-sb2z"
server_key: "677edc11da7d9e7a655c0208524d853e"
server_key_iv: "2a443af0d9e7172b146f37a68347a35e"
server_key_version: "encryptionkey@0"
I20260812 17:53:04.767287 15905 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 17:53:04.770083 15928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:53:04.771620 15905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:53:04.771781 15905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "9d0fa65e13e74406ba593485c46d7b4b"
format_stamp: "Formatted at 2026-08-12 17:53:04 on dist-test-slave-sb2z"
server_key: "677edc11da7d9e7a655c0208524d853e"
server_key_iv: "2a443af0d9e7172b146f37a68347a35e"
server_key_version: "encryptionkey@0"
I20260812 17:53:04.771915 15905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:53:04.777055 15905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:53:04.780858 15905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:53:04.781131 15905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:53:04.789417 15905 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:39429
I20260812 17:53:04.789440 16013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:39429 every 8 connection(s)
I20260812 17:53:04.790433 15905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:53:04.793253 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 15905
I20260812 17:53:04.793372 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:53:04.793574 16014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:53:04.793651 13464 external_mini_cluster.cc:1428] Setting key 4d54f63bf057b4504f7628227867af14
I20260812 17:53:04.799381 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:04.795129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53523 (local address 127.13.38.62:39429)
0812 17:53:04.795678 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:04.795694 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:04.795732 (+ 38us) server_negotiation.cc:407] Connection header received
0812 17:53:04.796503 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:04.796524 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:04.796824 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:04.797189 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:04.798094 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:04.798099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:04.798111 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:53:04.798122 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:04.798180 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:53:04.798199 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:04.798361 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:04.798438 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 17:53:04.798611 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":163,"threads_started":1}
I20260812 17:53:04.800990 16014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b: Bootstrap starting.
I20260812 17:53:04.802433 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:53:04.803720 16014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b: Neither blocks nor log segments found. Creating new log.
I20260812 17:53:04.804540 16014 log.cc:824] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b: Log is configured to *not* fsync() on all Append() calls
I20260812 17:53:04.806718 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:04.804722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37381 (local address 127.13.38.62:39429)
0812 17:53:04.804910 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:04.804914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:04.805036 (+ 122us) server_negotiation.cc:407] Connection header received
0812 17:53:04.805174 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:04.805177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:04.805215 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:04.805321 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:04.806251 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:04.806254 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:04.806257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 17:53:04.806267 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:04.806294 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 17:53:04.806303 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:04.806374 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:04.806518 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 17:53:04.806553 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260812 17:53:04.807358 16014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b: No bootstrap required, opened a new log
I20260812 17:53:04.811861 16014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0fa65e13e74406ba593485c46d7b4b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39429 } }
I20260812 17:53:04.812198 16014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 17:53:04.812284 16014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d0fa65e13e74406ba593485c46d7b4b, State: Initialized, Role: FOLLOWER
I20260812 17:53:04.812780 16014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0fa65e13e74406ba593485c46d7b4b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39429 } }
I20260812 17:53:04.812924 16014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 17:53:04.812971 16014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 17:53:04.813054 16014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:53:04.814088 16014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0fa65e13e74406ba593485c46d7b4b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39429 } }
I20260812 17:53:04.814481 16014 leader_election.cc:302] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d0fa65e13e74406ba593485c46d7b4b; no voters:
I20260812 17:53:04.814810 16014 leader_election.cc:288] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 17:53:04.814926 16030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:53:04.815228 16030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [term 1 LEADER]: Becoming Leader. State: Replica: 9d0fa65e13e74406ba593485c46d7b4b, State: Running, Role: LEADER
I20260812 17:53:04.815685 16030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0fa65e13e74406ba593485c46d7b4b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39429 } }
I20260812 17:53:04.816478 16014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:53:04.817677 16034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d0fa65e13e74406ba593485c46d7b4b. Latest consensus state: current_term: 1 leader_uuid: "9d0fa65e13e74406ba593485c46d7b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0fa65e13e74406ba593485c46d7b4b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39429 } } }
I20260812 17:53:04.817929 16032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d0fa65e13e74406ba593485c46d7b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0fa65e13e74406ba593485c46d7b4b" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 39429 } } }
I20260812 17:53:04.818050 16034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [sys.catalog]: This master's current role is: LEADER
I20260812 17:53:04.818084 16032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b [sys.catalog]: This master's current role is: LEADER
I20260812 17:53:04.818629 16050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:53:04.821765 16050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:53:04.828019 16050 catalog_manager.cc:1347] Generated new cluster ID: 84a31733cf264a149fa21a854629851f
I20260812 17:53:04.828179 16050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:53:04.849690 16050 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:53:04.850080 16050 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:53:04.859871 16050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d0fa65e13e74406ba593485c46d7b4b: Generated new TSK 0
I20260812 17:53:04.860708 16050 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 17:53:04.870560 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 15905
2026-08-12T17:53:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1699 ms)
[----------] 18 tests from AuthTokenIssuingTest (31180 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:53:04 dist-test-slave-sb2z krb5kdc[16067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:53:04 dist-test-slave-sb2z krb5kdc[16067](info): set up 2 sockets
Aug 12 17:53:04 dist-test-slave-sb2z krb5kdc[16067](info): commencing operation
krb5kdc: starting...
W20260812 17:53:06.159755 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:53:06 dist-test-slave-sb2z krb5kdc[16067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:53:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:53:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:53:06.304448 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:42429
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:42429,127.13.38.61:32931,127.13.38.60:35743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:53:06.425920 16094 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 17:53:06.426318 16094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:53:06.426404 16094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:53:06.426489 16094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:53:06.430766 16094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:53:06.430887 16094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:53:06.430930 16094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:53:06.430968 16094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:53:06.431000 16094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:53:06.436692 16094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:42429,127.13.38.61:32931,127.13.38.60:35743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:42429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:53:06.437911 16094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:53:06.439029 16094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:53:06.448559 16103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:06.448578 16106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:06.448580 16100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:53:06.449086 16094 server_base.cc:1034] running on GCE node
I20260812 17:53:06.449725 16094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:53:06.450925 16094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:53:06.452169 16094 hybrid_clock.cc:648] HybridClock initialized: now 1786557186452136 us; error 46 us; skew 500 ppm
Aug 12 17:53:06 dist-test-slave-sb2z krb5kdc[16067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:53:06.455432 16094 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:53:06.456533 16094 webserver.cc:460] Webserver started at http://127.13.38.62:34161/ using document root <none> and password file <none>
I20260812 17:53:06.457209 16094 fs_manager.cc:359] Metadata directory not provided
I20260812 17:53:06.457319 16094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:53:06.457592 16094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:53:06.460124 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "f1ba518c90f34caabbce78c4ba4a35a7"
format_stamp: "Formatted at 2026-08-12 17:53:06 on dist-test-slave-sb2z"
server_key: "46f663a68736bfa6a3a732fcf80f68b3"
server_key_iv: "fdca44d12d2bc02863ad75275575e28c"
server_key_version: "encryptionkey@0"
I20260812 17:53:06.460786 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "f1ba518c90f34caabbce78c4ba4a35a7"
format_stamp: "Formatted at 2026-08-12 17:53:06 on dist-test-slave-sb2z"
server_key: "46f663a68736bfa6a3a732fcf80f68b3"
server_key_iv: "fdca44d12d2bc02863ad75275575e28c"
server_key_version: "encryptionkey@0"
I20260812 17:53:06.465147 16094 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:53:06.468037 16118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:53:06.469520 16094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:53:06.469694 16094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "f1ba518c90f34caabbce78c4ba4a35a7"
format_stamp: "Formatted at 2026-08-12 17:53:06 on dist-test-slave-sb2z"
server_key: "46f663a68736bfa6a3a732fcf80f68b3"
server_key_iv: "fdca44d12d2bc02863ad75275575e28c"
server_key_version: "encryptionkey@0"
I20260812 17:53:06.469830 16094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:53:06.490469 16094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:53:06.493942 16094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:53:06.494182 16094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:53:06.502373 16094 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:42429
I20260812 17:53:06.502692 16203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:42429 every 8 connection(s)
I20260812 17:53:06.503495 16094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:53:06.506559 16204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:53:06.510761 16204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:06.513423 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 16094
I20260812 17:53:06.513620 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:53:06.513952 13464 external_mini_cluster.cc:1428] Setting key 6cdc498cad1c958c898d18d6d2254299
Aug 12 17:53:06 dist-test-slave-sb2z krb5kdc[16067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.38.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.61@KRBTEST.COM" created.
I20260812 17:53:06.530119 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.513435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42429 (local address 127.0.0.1:40910)
0812 17:53:06.514315 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.514377 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:06.515800 (+ 1423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:06.516792 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:06.516797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:06.517340 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:06.517654 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:06.517670 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.518916 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.518922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:06.520315 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:06.520332 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.521188 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.521195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:06.521363 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:06.522071 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:06.522112 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:06.524126 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:06.526511 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:06.526521 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:06.526531 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:06.526818 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:06.527237 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:06.527240 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:06.527241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:06.527361 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:06.528213 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:06.528220 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:06.528386 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 17:53:06.528601 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 17:53:06.529022 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":119,"thread_start_us":168,"threads_started":1}
I20260812 17:53:06.530119 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.513435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40910 (local address 127.13.38.62:42429)
0812 17:53:06.514012 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:06.514034 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:06.514514 (+ 480us) server_negotiation.cc:407] Connection header received
0812 17:53:06.516218 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:06.516226 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:06.516517 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:06.516782 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:06.517832 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.518769 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.520479 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.520976 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.521127 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:06.524338 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:06.524345 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:06.524358 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:06.524384 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:06.526367 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:06.526967 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:06.526972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:06.526974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:06.527132 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:06.527456 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:06.527459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:06.527460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:06.528050 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:06.528251 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:06.528660 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 17:53:06.528969 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":394,"spinlock_wait_cycles":26368,"thread_start_us":169,"threads_started":1}
I20260812 17:53:06.532655 16204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:06.533548 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.533258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:32931 (local address 127.0.0.1:37652)
0812 17:53:06.533396 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.533475 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:32931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 17:53:06.533782 16119 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.38.61:32931: connect: Connection refused (error 111)
I20260812 17:53:06.534729 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.534487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:32931 (local address 127.0.0.1:37656)
0812 17:53:06.534613 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.534669 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:32931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 17:53:06.534907 16204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.61:32931: Network error: Client connection negotiation failed: client connection to 127.13.38.61:32931: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.61.keytab.
Entry for principal kudu/127.13.38.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.61.keytab.
I20260812 17:53:06.547478 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.61:32931
--webserver_interface=127.13.38.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.61.keytab
--principal=kudu/127.13.38.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:42429,127.13.38.61:32931,127.13.38.60:35743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:53:06.584177 16204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } attempt: 1
I20260812 17:53:06.585193 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.584867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:32931 (local address 127.0.0.1:37664)
0812 17:53:06.585046 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.585113 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:32931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 17:53:06.585983 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.585784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:32931 (local address 127.0.0.1:37678)
0812 17:53:06.585882 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.585927 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:32931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 17:53:06.586148 16204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.61:32931: Network error: Client connection negotiation failed: client connection to 127.13.38.61:32931: connect: Connection refused (error 111)
I20260812 17:53:06.654340 16204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } attempt: 2
I20260812 17:53:06.655344 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.654976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:32931 (local address 127.0.0.1:37682)
0812 17:53:06.655193 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.655260 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:32931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260812 17:53:06.656243 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.655999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:32931 (local address 127.0.0.1:37692)
0812 17:53:06.656134 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.656182 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:32931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 17:53:06.656484 16204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.61:32931: Network error: Client connection negotiation failed: client connection to 127.13.38.61:32931: connect: Connection refused (error 111)
W20260812 17:53:06.669476 16215 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 17:53:06.669816 16215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:53:06.669842 16215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:53:06.669946 16215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:53:06.674348 16215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:53:06.674422 16215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:53:06.674443 16215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:53:06.674460 16215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:53:06.674476 16215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:53:06.680191 16215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:42429,127.13.38.61:32931,127.13.38.60:35743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.61.keytab
--principal=kudu/127.13.38.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.61:32931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.38.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:53:06.681492 16215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:53:06.682624 16215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:53:06.692102 16226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:06.692097 16222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:06.692137 16223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:53:06.692418 16215 server_base.cc:1034] running on GCE node
I20260812 17:53:06.693217 16215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:53:06.694444 16215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:53:06.695663 16215 hybrid_clock.cc:648] HybridClock initialized: now 1786557186695625 us; error 50 us; skew 500 ppm
Aug 12 17:53:06 dist-test-slave-sb2z krb5kdc[16067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:53:06.703893 16215 init.cc:377] Logged in from keytab as kudu/127.13.38.61@KRBTEST.COM (short username kudu)
I20260812 17:53:06.705070 16215 webserver.cc:460] Webserver started at http://127.13.38.61:39127/ using document root <none> and password file <none>
I20260812 17:53:06.705724 16215 fs_manager.cc:359] Metadata directory not provided
I20260812 17:53:06.705826 16215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:53:06.706099 16215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:53:06.708492 16215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/data/instance:
uuid: "cc63c0536baa470fbf77c9a499ce8e9c"
format_stamp: "Formatted at 2026-08-12 17:53:06 on dist-test-slave-sb2z"
server_key: "1a06e57020adab119d81292190c997c9"
server_key_iv: "c551d19de599b8f0431e988c508e0c53"
server_key_version: "encryptionkey@0"
I20260812 17:53:06.709120 16215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/wal/instance:
uuid: "cc63c0536baa470fbf77c9a499ce8e9c"
format_stamp: "Formatted at 2026-08-12 17:53:06 on dist-test-slave-sb2z"
server_key: "1a06e57020adab119d81292190c997c9"
server_key_iv: "c551d19de599b8f0431e988c508e0c53"
server_key_version: "encryptionkey@0"
I20260812 17:53:06.713201 16215 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:53:06.715834 16234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:53:06.717159 16215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 17:53:06.717312 16215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/wal
uuid: "cc63c0536baa470fbf77c9a499ce8e9c"
format_stamp: "Formatted at 2026-08-12 17:53:06 on dist-test-slave-sb2z"
server_key: "1a06e57020adab119d81292190c997c9"
server_key_iv: "c551d19de599b8f0431e988c508e0c53"
server_key_version: "encryptionkey@0"
I20260812 17:53:06.717435 16215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:53:06.730404 16215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:53:06.733944 16215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:53:06.734210 16215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:53:06.742178 16215 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.61:32931
I20260812 17:53:06.742496 16321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.61:32931 every 8 connection(s)
I20260812 17:53:06.743290 16215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
I20260812 17:53:06.746014 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 16215
I20260812 17:53:06.746176 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-1/wal/instance
I20260812 17:53:06.746500 13464 external_mini_cluster.cc:1428] Setting key 302ccf5a0a87813bb7ab030bbae3bde3
I20260812 17:53:06.747879 16323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:53:06.748778 16204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } attempt: 3
I20260812 17:53:06.753283 16323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.38.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.60@KRBTEST.COM" created.
Aug 12 17:53:06 dist-test-slave-sb2z krb5kdc[16067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for kudu/127.13.38.61@KRBTEST.COM
Aug 12 17:53:06 dist-test-slave-sb2z krb5kdc[16067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.61@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:53:06.771617 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.749409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:32931 (local address 127.0.0.1:37708)
0812 17:53:06.749646 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.749663 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:06.749763 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:06.754320 (+ 4557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:06.754324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:06.754546 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:06.754707 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:06.754715 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.757047 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.757066 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:06.758243 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:06.758262 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.759573 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.759580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:06.759729 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:06.760903 (+ 1174us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:06.760925 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:06.764623 (+ 3698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:06.768888 (+ 4265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:06.768892 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:06.768897 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:06.769224 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:06.769680 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:06.769686 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:06.769689 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:06.769798 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:06.770692 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:06.770699 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:06.770880 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 17:53:06.771041 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 17:53:06.771263 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260812 17:53:06.772657 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.755427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42429 (local address 127.0.0.1:40926)
0812 17:53:06.756661 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.756731 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:06.756900 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:06.757286 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:06.757295 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:06.757875 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:06.758100 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:06.758110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.759432 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.759437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:06.760699 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:06.760718 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.761723 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.761730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:06.761854 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:06.762638 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:06.762683 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:06.765608 (+ 2925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:06.768047 (+ 2439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:06.768063 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:06.768080 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:06.768716 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:06.769116 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:06.769122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:06.769128 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:06.769353 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:06.770030 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:06.770042 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:06.770512 (+ 470us) client_negotiation.cc:769] Sending connection context
0812 17:53:06.770741 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 17:53:06.771178 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"thread_start_us":854,"threads_started":1}
I20260812 17:53:06.773414 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.750740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.13.38.61:32931)
0812 17:53:06.751735 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:06.751784 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:06.751861 (+ 77us) server_negotiation.cc:407] Connection header received
0812 17:53:06.753151 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:06.753203 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:06.753894 (+ 691us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:06.754936 (+ 1042us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:06.755393 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.756876 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.758439 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.759384 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.760141 (+ 757us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:06.765156 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:06.765168 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:06.765192 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:06.765233 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:06.768712 (+ 3479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:06.769410 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:06.769419 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:06.769421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:06.769527 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:06.769957 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:06.769961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:06.769963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:06.770518 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:06.772564 (+ 2046us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:06.772734 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 17:53:06.772987 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":152,"threads_started":1}
I20260812 17:53:06.774727 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.755544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40926 (local address 127.13.38.62:42429)
0812 17:53:06.755726 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:06.755734 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:06.756824 (+ 1090us) server_negotiation.cc:407] Connection header received
0812 17:53:06.757027 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:06.757034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:06.757095 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:06.757222 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:06.758285 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.759198 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.760944 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.761465 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.761707 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:06.765771 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:06.765777 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:06.765782 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:06.765820 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:06.767900 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:06.768868 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:06.768875 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:06.768881 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:06.768993 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:06.769499 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:06.769506 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:06.769512 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:06.769821 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:06.769946 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:06.774138 (+ 4192us) server_negotiation.cc:299] Negotiation successful
0812 17:53:06.774352 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 17:53:06.776197 16323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:06.777055 16204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:06.777911 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.777567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51218)
0812 17:53:06.777780 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.777845 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260812 17:53:06.778987 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.778553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51220)
0812 17:53:06.778810 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.778909 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260812 17:53:06.779135 16204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:35743: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Aug 12 17:53:06 dist-test-slave-sb2z krb5kdc[16067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.61@KRBTEST.COM for kudu/127.13.38.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.60.keytab.
Entry for principal kudu/127.13.38.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.60.keytab.
I20260812 17:53:06.787626 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.60:35743
--webserver_interface=127.13.38.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.60.keytab
--principal=kudu/127.13.38.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:42429,127.13.38.61:32931,127.13.38.60:35743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:53:06.789351 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.776772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:32931 (local address 127.0.0.1:37714)
0812 17:53:06.776895 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.776912 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:06.777006 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:06.777749 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:06.777753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:06.778025 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:06.778171 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:06.778182 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.779428 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.779432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:06.780354 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:06.780368 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.781269 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.781271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:06.781337 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:06.782020 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:06.782041 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:06.784338 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:06.787124 (+ 2786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:06.787129 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:06.787132 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:06.787451 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:06.787879 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:06.787883 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:06.787886 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:06.788006 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:06.788560 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:06.788564 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:06.788707 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 17:53:06.788867 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 17:53:06.789057 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 17:53:06.789719 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.777012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37714 (local address 127.13.38.61:32931)
0812 17:53:06.777142 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:06.777148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:06.777162 (+ 14us) server_negotiation.cc:407] Connection header received
0812 17:53:06.777203 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:06.777207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:06.777259 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:06.777381 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:06.778339 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.779212 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.780542 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:06.780957 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:06.781126 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:06.784543 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:06.784547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:06.784550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:06.784612 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:06.786970 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:06.787651 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:06.787658 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:06.787660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:06.787768 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:06.788136 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:06.788142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:06.788147 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:06.788450 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:06.788567 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:06.788945 (+ 378us) server_negotiation.cc:299] Negotiation successful
0812 17:53:06.789184 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":54}
I20260812 17:53:06.793298 16323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:06.794293 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.793868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51236)
0812 17:53:06.794066 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.794191 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260812 17:53:06.795166 16236 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
I20260812 17:53:06.796310 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.796064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51240)
0812 17:53:06.796172 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.796226 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 17:53:06.796512 16323 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:35743: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
I20260812 17:53:06.810362 16204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } attempt: 1
I20260812 17:53:06.811362 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.810999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51244)
0812 17:53:06.811187 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.811265 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 17:53:06.812284 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.811955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51252)
0812 17:53:06.812165 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.812216 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260812 17:53:06.812453 16204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:35743: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
I20260812 17:53:06.845813 16323 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } attempt: 1
I20260812 17:53:06.846874 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.846501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51266)
0812 17:53:06.846694 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.846772 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 17:53:06.847730 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.847451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51268)
0812 17:53:06.847603 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.847661 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 17:53:06.847877 16323 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:35743: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
I20260812 17:53:06.887636 16204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } attempt: 2
I20260812 17:53:06.888641 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.888286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51270)
0812 17:53:06.888471 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.888545 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 17:53:06.889521 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.889279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51272)
0812 17:53:06.889398 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.889454 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 17:53:06.889660 16204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:35743: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
I20260812 17:53:06.916108 16323 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } attempt: 2
I20260812 17:53:06.917062 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.916667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51282)
0812 17:53:06.916850 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.916979 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 17:53:06.918118 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.917773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51298)
0812 17:53:06.917967 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.918036 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 17:53:06.918308 16323 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:35743: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
W20260812 17:53:06.920708 16334 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 17:53:06.921073 16334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:53:06.921137 16334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:53:06.921217 16334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:53:06.925541 16334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:53:06.925642 16334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:53:06.925678 16334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:53:06.925715 16334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:53:06.925753 16334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:53:06.931411 16334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:42429,127.13.38.61:32931,127.13.38.60:35743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.60.keytab
--principal=kudu/127.13.38.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.60:35743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.38.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:53:06.932685 16334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:53:06.933815 16334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:53:06.943087 16342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:06.943107 16344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:06.943096 16346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:53:06.943161 16334 server_base.cc:1034] running on GCE node
I20260812 17:53:06.944061 16334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:53:06.945293 16334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:53:06.946488 16334 hybrid_clock.cc:648] HybridClock initialized: now 1786557186946471 us; error 30 us; skew 500 ppm
Aug 12 17:53:06 dist-test-slave-sb2z krb5kdc[16067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:53:06.949864 16334 init.cc:377] Logged in from keytab as kudu/127.13.38.60@KRBTEST.COM (short username kudu)
I20260812 17:53:06.950838 16334 webserver.cc:460] Webserver started at http://127.13.38.60:45101/ using document root <none> and password file <none>
I20260812 17:53:06.951440 16334 fs_manager.cc:359] Metadata directory not provided
I20260812 17:53:06.951505 16334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:53:06.951822 16334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:53:06.954025 16334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/data/instance:
uuid: "a1a74f93655d4be2ab9155a63a2c264d"
format_stamp: "Formatted at 2026-08-12 17:53:06 on dist-test-slave-sb2z"
server_key: "b76e6e53a2f549e8e5288fc113e2e9e4"
server_key_iv: "8dd7d0ab1e7fb91d41f685304a7aace3"
server_key_version: "encryptionkey@0"
I20260812 17:53:06.954664 16334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/wal/instance:
uuid: "a1a74f93655d4be2ab9155a63a2c264d"
format_stamp: "Formatted at 2026-08-12 17:53:06 on dist-test-slave-sb2z"
server_key: "b76e6e53a2f549e8e5288fc113e2e9e4"
server_key_iv: "8dd7d0ab1e7fb91d41f685304a7aace3"
server_key_version: "encryptionkey@0"
I20260812 17:53:06.959132 16334 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 17:53:06.961838 16353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:53:06.963135 16334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 17:53:06.963392 16334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/wal
uuid: "a1a74f93655d4be2ab9155a63a2c264d"
format_stamp: "Formatted at 2026-08-12 17:53:06 on dist-test-slave-sb2z"
server_key: "b76e6e53a2f549e8e5288fc113e2e9e4"
server_key_iv: "8dd7d0ab1e7fb91d41f685304a7aace3"
server_key_version: "encryptionkey@0"
I20260812 17:53:06.963573 16334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:53:06.985639 16334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:53:06.988837 16204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } attempt: 3
I20260812 17:53:06.989789 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.989407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51310)
0812 17:53:06.989603 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.989685 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 17:53:06.989869 16334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:53:06.990094 16334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:53:06.990612 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:06.990365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51316)
0812 17:53:06.990490 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:06.990546 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 17:53:06.990777 16204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:35743: Network error: Client connection negotiation failed: client connection to 127.13.38.60:35743: connect: Connection refused (error 111)
I20260812 17:53:06.997836 16334 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.60:35743
I20260812 17:53:06.998178 16432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.60:35743 every 8 connection(s)
I20260812 17:53:06.998958 16334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
I20260812 17:53:07.001969 16433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:53:07.005983 16433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:07.009198 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 16334
I20260812 17:53:07.009341 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786556880371321-13464-0/minicluster-data/master-2/wal/instance
I20260812 17:53:07.009503 16323 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } attempt: 3
I20260812 17:53:07.009676 13464 external_mini_cluster.cc:1428] Setting key 9d44447988df63c2cf02a5eb39c8c3ce
Aug 12 17:53:07 dist-test-slave-sb2z krb5kdc[16067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.61@KRBTEST.COM for kudu/127.13.38.60@KRBTEST.COM
Aug 12 17:53:07 dist-test-slave-sb2z krb5kdc[16067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.60@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
Aug 12 17:53:07 dist-test-slave-sb2z krb5kdc[16067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.60@KRBTEST.COM
I20260812 17:53:07.022725 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.009988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51318)
0812 17:53:07.010173 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:07.010187 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:07.010289 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:07.011322 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:07.011326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:07.011561 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:07.011681 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:07.011691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.013638 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.013641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:07.014361 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:07.014373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.015142 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.015146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:07.015254 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.015915 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:07.015930 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:07.017551 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:07.020207 (+ 2656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:07.020211 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:07.020216 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:07.020485 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:07.021029 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:07.021033 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:07.021036 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:07.021157 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:07.022129 (+ 972us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:07.022132 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:07.022237 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 17:53:07.022358 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 17:53:07.022531 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 17:53:07.022764 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.008108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40942 (local address 127.13.38.62:42429)
0812 17:53:07.008254 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.008257 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.009232 (+ 975us) server_negotiation.cc:407] Connection header received
0812 17:53:07.010496 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.010500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.010561 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.010625 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.012378 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.012961 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.014214 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.014567 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.014676 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.018570 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:07.018574 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:07.018576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:07.018610 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:07.020457 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.021217 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.021221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.021223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.021317 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.021634 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.021638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.021639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.021862 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:07.021987 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:07.022511 (+ 524us) server_negotiation.cc:299] Negotiation successful
0812 17:53:07.022633 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:53:07.023859 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.010142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51318 (local address 127.13.38.60:35743)
0812 17:53:07.010653 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.010667 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.010714 (+ 47us) server_negotiation.cc:407] Connection header received
0812 17:53:07.010883 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.010890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.011146 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.011334 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.011961 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.013500 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.014478 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.014972 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.015487 (+ 515us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.017713 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:07.017720 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:07.017736 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:07.017768 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:07.020069 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.020622 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.020631 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.020641 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.020838 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.021288 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.021292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.021293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.021924 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:07.022152 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:07.022479 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 17:53:07.022812 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":270,"server-negotiator.queue_time_us":405,"spinlock_wait_cycles":44544,"thread_start_us":175,"threads_started":1}
I20260812 17:53:07.023859 16436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.008567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:42429 (local address 127.0.0.1:40942)
0812 17:53:07.009047 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:07.009120 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:07.010248 (+ 1128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:07.010979 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:07.010988 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:07.011488 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:07.012215 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:07.012229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.013090 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.013099 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:07.014056 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:07.014066 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.014718 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.014724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:07.014910 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.015867 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:07.015892 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:07.018401 (+ 2509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:07.020653 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:07.020660 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:07.020667 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:07.021115 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:07.021432 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:07.021442 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:07.021444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:07.021536 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:07.021984 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:07.021992 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:07.022188 (+ 196us) client_negotiation.cc:769] Sending connection context
0812 17:53:07.022959 (+ 771us) client_negotiation.cc:240] Negotiation successful
0812 17:53:07.023153 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":90,"spinlock_wait_cycles":896,"thread_start_us":121,"threads_started":1}
I20260812 17:53:07.024227 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.010516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51334 (local address 127.13.38.60:35743)
0812 17:53:07.010837 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.010853 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.010866 (+ 13us) server_negotiation.cc:407] Connection header received
0812 17:53:07.010973 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.010976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.011147 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.011335 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.012583 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.013500 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.014503 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.014971 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.015096 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.018735 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:07.018741 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:07.018746 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:07.018777 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:07.021534 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.022055 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.022060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.022065 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.022152 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.022463 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.022466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.022467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.022650 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:07.023856 (+ 1206us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:07.023944 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 17:53:07.024061 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":170,"server-negotiator.queue_time_us":258,"spinlock_wait_cycles":36736,"thread_start_us":109,"threads_started":1}
I20260812 17:53:07.025955 16433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:07.029256 16323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c: Bootstrap starting.
Aug 12 17:53:07 dist-test-slave-sb2z krb5kdc[16067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:53:07.032526 16323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c: Neither blocks nor log segments found. Creating new log.
Aug 12 17:53:07 dist-test-slave-sb2z krb5kdc[16067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.60@KRBTEST.COM for kudu/127.13.38.61@KRBTEST.COM
I20260812 17:53:07.033567 16323 log.cc:824] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c: Log is configured to *not* fsync() on all Append() calls
I20260812 17:53:07.036271 16323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c: No bootstrap required, opened a new log
I20260812 17:53:07.037282 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.027946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.13.38.61:32931)
0812 17:53:07.028097 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.028102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.028462 (+ 360us) server_negotiation.cc:407] Connection header received
0812 17:53:07.028576 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.028579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.028621 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.028719 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.029138 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.029837 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.030758 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.030993 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.031103 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.033371 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:07.033375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:07.033378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:07.033404 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:07.035228 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.035797 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.035801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.035803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.035897 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.036249 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.036252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.036254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.036518 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:07.036596 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:07.036981 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 17:53:07.037126 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:53:07.037696 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.026337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40952 (local address 127.13.38.62:42429)
0812 17:53:07.026461 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.026464 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.026476 (+ 12us) server_negotiation.cc:407] Connection header received
0812 17:53:07.026572 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.026574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.026610 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.026674 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.027482 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.028203 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.029241 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.029625 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.029728 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.033519 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:07.033525 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:07.033532 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:07.033562 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:07.035479 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.036076 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.036080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.036082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.036179 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.036483 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.036487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.036489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.036701 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:07.036952 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:07.037087 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 17:53:07.037220 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 17:53:07.037727 16436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.027774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:32931 (local address 127.0.0.1:37720)
0812 17:53:07.028366 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:07.028390 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:07.028498 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:07.028734 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:07.028737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:07.028916 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:07.029023 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:07.029028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.029961 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.029967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:07.030630 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:07.030641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.031116 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.031119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:07.031225 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.031738 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:07.031750 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:07.033215 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:07.035371 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:07.035377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:07.035380 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:07.035675 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:07.036013 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:07.036017 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:07.036019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:07.036126 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:07.036687 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:07.036690 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:07.036790 (+ 100us) client_negotiation.cc:769] Sending connection context
0812 17:53:07.037420 (+ 630us) client_negotiation.cc:240] Negotiation successful
0812 17:53:07.037569 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524}
I20260812 17:53:07.038769 16433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:07.041297 16323 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } }
I20260812 17:53:07.041554 16323 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc63c0536baa470fbf77c9a499ce8e9c, State: Initialized, Role: FOLLOWER
I20260812 17:53:07.041923 16323 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } }
I20260812 17:53:07.042512 16447 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } } }
I20260812 17:53:07.042758 16447 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:53:07.043613 16323 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [sys.catalog]: configured and running, proceeding with master startup.
Aug 12 17:53:07 dist-test-slave-sb2z krb5kdc[16067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.60@KRBTEST.COM for kudu/127.13.38.60@KRBTEST.COM
Aug 12 17:53:07 dist-test-slave-sb2z krb5kdc[16067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.61@KRBTEST.COM
I20260812 17:53:07.049144 16436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.039319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51340)
0812 17:53:07.039670 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:07.039682 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:07.039770 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:07.040027 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:07.040030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:07.040206 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:07.040305 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:07.040310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.041293 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.041295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:07.042020 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:07.042033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.042682 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.042685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:07.042752 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.043309 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:07.043322 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:07.045229 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:07.047323 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:07.047329 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:07.047334 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:07.047645 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:07.047947 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:07.047950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:07.047952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:07.048037 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:07.048650 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:07.048653 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:07.048744 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 17:53:07.048837 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 17:53:07.048981 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305}
W20260812 17:53:07.049629 16469 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:53:07.049712 16469 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:53:07.050562 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.039258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37722 (local address 127.13.38.61:32931)
0812 17:53:07.039370 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.039373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.039383 (+ 10us) server_negotiation.cc:407] Connection header received
0812 17:53:07.039410 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.039412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.039444 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.039531 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.040310 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.040823 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.041674 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.041916 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.041978 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.045457 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:07.045464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:07.045468 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:07.045495 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:07.048319 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.048832 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.048838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.048840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.048927 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.049378 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.049382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.049383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.049623 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:07.049865 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:07.050286 (+ 421us) server_negotiation.cc:299] Negotiation successful
0812 17:53:07.050412 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 17:53:07.051968 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.039492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51340 (local address 127.13.38.60:35743)
0812 17:53:07.039608 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.039612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.039757 (+ 145us) server_negotiation.cc:407] Connection header received
0812 17:53:07.039847 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.039850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.039903 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.039995 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.040427 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.041168 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.042163 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.042484 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.042556 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.045392 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:07.045398 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:07.045403 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:07.045431 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:07.047190 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.047764 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.047767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.047769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.047854 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.048189 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.048193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.048194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.048509 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:07.048613 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:07.051665 (+ 3052us) server_negotiation.cc:299] Negotiation successful
0812 17:53:07.051798 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 17:53:07.047572 16215 master_runner.cc:186] Error getting master registration for 127.13.38.62:42429: OK, instance_id { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" instance_seqno: 1786557186500225 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 17:53:07.052685 16215 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 17:53:07.054490 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:53:07.055136 16433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d: Bootstrap starting.
I20260812 17:53:07.056049 16156 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f1ba518c90f34caabbce78c4ba4a35a7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40952
I20260812 17:53:07.057556 16433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d: Neither blocks nor log segments found. Creating new log.
I20260812 17:53:07.058270 16433 log.cc:824] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d: Log is configured to *not* fsync() on all Append() calls
I20260812 17:53:07.060290 16433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d: No bootstrap required, opened a new log
I20260812 17:53:07.063313 16433 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } }
I20260812 17:53:07.063580 16433 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1a74f93655d4be2ab9155a63a2c264d, State: Initialized, Role: FOLLOWER
I20260812 17:53:07.063895 16433 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } }
I20260812 17:53:07.064524 16472 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } } }
I20260812 17:53:07.064846 16472 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:53:07.065085 16433 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [sys.catalog]: configured and running, proceeding with master startup.
W20260812 17:53:07.071514 16487 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:53:07.071741 16487 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:53:07.069123 16334 master_runner.cc:186] Error getting master registration for 127.13.38.62:42429: OK, instance_id { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" instance_seqno: 1786557186500225 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 17:53:07.072788 16334 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 17:53:07.154914 16204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } attempt: 4
Aug 12 17:53:07 dist-test-slave-sb2z krb5kdc[16067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for kudu/127.13.38.60@KRBTEST.COM
I20260812 17:53:07.165802 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.155548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:35743 (local address 127.0.0.1:51342)
0812 17:53:07.155868 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:07.155888 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:07.156029 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:07.156413 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:07.156419 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:07.156671 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:07.156804 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:07.156812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.157868 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.157872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:07.158723 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:07.158739 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.159370 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.159374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:07.159462 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.160129 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:07.160147 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:07.161857 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:07.164068 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:07.164071 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:07.164074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:07.164345 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:07.164654 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:07.164657 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:07.164659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:07.164745 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:07.165250 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:07.165254 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:07.165360 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 17:53:07.165460 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 17:53:07.165617 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188}
I20260812 17:53:07.166409 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.155794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51342 (local address 127.13.38.60:35743)
0812 17:53:07.155937 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.155941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.155959 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:53:07.156193 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.156196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.156250 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.156339 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.156968 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.157751 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.158885 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.159252 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.159360 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.162054 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:07.162059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:07.162061 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:07.162093 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:07.163906 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.164458 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.164462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.164464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.164552 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.164842 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.164845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.164847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.165106 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:07.165203 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:07.166123 (+ 920us) server_negotiation.cc:299] Negotiation successful
0812 17:53:07.166253 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 17:53:07.169998 16204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7: Bootstrap starting.
I20260812 17:53:07.173252 16204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7: Neither blocks nor log segments found. Creating new log.
I20260812 17:53:07.174352 16204 log.cc:824] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7: Log is configured to *not* fsync() on all Append() calls
I20260812 17:53:07.176975 16204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7: No bootstrap required, opened a new log
I20260812 17:53:07.180307 16204 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } }
I20260812 17:53:07.180588 16204 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ba518c90f34caabbce78c4ba4a35a7, State: Initialized, Role: FOLLOWER
I20260812 17:53:07.180974 16204 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } }
I20260812 17:53:07.181679 16492 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } } }
I20260812 17:53:07.182109 16492 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:53:07.182512 16204 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:53:07.182600 16156 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 17:53:07.182724 16156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:53:07.184258 16156 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } }
I20260812 17:53:07.185400 16156 leader_election.cc:288] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [CANDIDATE]: Term 1 election: Requested vote from peers cc63c0536baa470fbf77c9a499ce8e9c (127.13.38.61:32931), a1a74f93655d4be2ab9155a63a2c264d (127.13.38.60:35743)
I20260812 17:53:07.186508 16280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cc63c0536baa470fbf77c9a499ce8e9c"
I20260812 17:53:07.186868 16280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:53:07.188050 16389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a1a74f93655d4be2ab9155a63a2c264d"
I20260812 17:53:07.188313 16389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:53:07.189709 16389 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ba518c90f34caabbce78c4ba4a35a7 in term 1.
W20260812 17:53:07.191753 16514 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:53:07.192252 16514 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:53:07.192101 16119 leader_election.cc:302] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1a74f93655d4be2ab9155a63a2c264d, f1ba518c90f34caabbce78c4ba4a35a7; no voters:
I20260812 17:53:07.192704 16492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:53:07.192924 16492 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [term 1 LEADER]: Becoming Leader. State: Replica: f1ba518c90f34caabbce78c4ba4a35a7, State: Running, Role: LEADER
I20260812 17:53:07.193279 16492 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } }
I20260812 17:53:07.195142 16492 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1ba518c90f34caabbce78c4ba4a35a7. Latest consensus state: current_term: 1 leader_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } } }
I20260812 17:53:07.195247 16492 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [sys.catalog]: This master's current role is: LEADER
I20260812 17:53:07.196624 16280 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ba518c90f34caabbce78c4ba4a35a7 in term 1.
I20260812 17:53:07.203898 16525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:53:07.205142 16525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:53:07.218658 16389 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [term 1 FOLLOWER]: Refusing update from remote peer f1ba518c90f34caabbce78c4ba4a35a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:53:07.219795 16492 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:53:07.220817 16280 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [term 1 FOLLOWER]: Refusing update from remote peer f1ba518c90f34caabbce78c4ba4a35a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:53:07.221567 16492 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:53:07.227032 16447 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1ba518c90f34caabbce78c4ba4a35a7. Latest consensus state: current_term: 1 leader_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } } }
I20260812 17:53:07.227190 16447 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:53:07.230438 16472 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1ba518c90f34caabbce78c4ba4a35a7. Latest consensus state: current_term: 1 leader_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } } }
I20260812 17:53:07.230590 16472 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:53:07.231767 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.197483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40956 (local address 127.13.38.62:42429)
0812 17:53:07.197605 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.197610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.203905 (+ 6295us) server_negotiation.cc:407] Connection header received
0812 17:53:07.204120 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.204123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.204174 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.204227 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.205340 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.206147 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.207386 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.207730 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.207815 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.209929 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:07.209934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:07.209937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:07.209970 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:07.222942 (+ 12972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.229480 (+ 6538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.229485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.229489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.229625 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.229983 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.229986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.229988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.230246 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:07.230363 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:07.231291 (+ 928us) server_negotiation.cc:299] Negotiation successful
0812 17:53:07.231473 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 17:53:07.236830 16492 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } } }
I20260812 17:53:07.237004 16492 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [sys.catalog]: This master's current role is: LEADER
W20260812 17:53:07.237195 16242 tablet.cc:2369] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:53:07.237834 16526 mvcc.cc:205] Tried to move back new op lower bound from 7317738238809497600 to 7317738238747164672. Current Snapshot: MvccSnapshot[applied={T|T < 7317738238809497600}]
I20260812 17:53:07.239296 16447 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } } }
I20260812 17:53:07.239444 16447 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:53:07.239579 16527 mvcc.cc:205] Tried to move back new op lower bound from 7317738238809497600 to 7317738238747164672. Current Snapshot: MvccSnapshot[applied={T|T < 7317738238809497600}]
W20260812 17:53:07.241904 16362 tablet.cc:2369] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:53:07.242133 16525 catalog_manager.cc:1347] Generated new cluster ID: f99b8c984f6041159cf227b31da2bd99
I20260812 17:53:07.242203 16525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:53:07.247578 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.198047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37730 (local address 127.13.38.61:32931)
0812 17:53:07.198216 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.198223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.230988 (+ 32765us) server_negotiation.cc:407] Connection header received
0812 17:53:07.231086 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.231091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.231150 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.231239 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.232222 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.232989 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.235192 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.235495 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.235604 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.242961 (+ 7357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:07.242966 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:07.242969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:07.243004 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:07.244903 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.245534 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.245538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.245540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.245663 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.246043 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.246046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.246048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.246291 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:07.246412 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:07.247229 (+ 817us) server_negotiation.cc:299] Negotiation successful
0812 17:53:07.247369 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 17:53:07.251495 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:07.198214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51348 (local address 127.13.38.60:35743)
0812 17:53:07.198360 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.198364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.232673 (+ 34309us) server_negotiation.cc:407] Connection header received
0812 17:53:07.232944 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.232948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.233003 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.233056 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.234058 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.234826 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.236006 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:07.236330 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:07.236409 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:07.238728 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:07.238732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:07.238735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:07.238770 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:07.241502 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.249746 (+ 8244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.249751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.249753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.249882 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:07.250334 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:07.250338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:07.250340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:07.250615 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:07.250766 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:07.251175 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 17:53:07.251312 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 17:53:07.254216 16525 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:53:07.255084 16520 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } } }
I20260812 17:53:07.255214 16520 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7 [sys.catalog]: This master's current role is: LEADER
I20260812 17:53:07.255721 16525 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:53:07.256805 16472 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ba518c90f34caabbce78c4ba4a35a7" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 42429 } } peers { permanent_uuid: "cc63c0536baa470fbf77c9a499ce8e9c" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 32931 } } peers { permanent_uuid: "a1a74f93655d4be2ab9155a63a2c264d" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 35743 } } }
I20260812 17:53:07.256914 16472 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:53:07.258553 16532 mvcc.cc:205] Tried to move back new op lower bound from 7317738238809497600 to 7317738238747164672. Current Snapshot: MvccSnapshot[applied={T|T < 7317738238809497600}]
I20260812 17:53:07.275082 16525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1ba518c90f34caabbce78c4ba4a35a7: Generated new TSK 0
I20260812 17:53:07.276196 16525 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 17:53:07.315428 16210 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.313831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40962 (local address 127.13.38.62:42429)
0812 17:53:07.313962 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.313968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.314289 (+ 321us) server_negotiation.cc:407] Connection header received
0812 17:53:07.314479 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.314482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.314533 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.314588 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.315278 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40962: BlockingRecv error: recv got EOF from 127.0.0.1:40962 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 17:53:07.315488 16328 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.314110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37736 (local address 127.13.38.61:32931)
0812 17:53:07.314249 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.314253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.314268 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:53:07.314433 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.314439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.314504 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.314572 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.315317 (+ 745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37736: BlockingRecv error: recv got EOF from 127.0.0.1:37736 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 17:53:07.321254 16442 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.313831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51364 (local address 127.13.38.60:35743)
0812 17:53:07.313969 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.313974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.315825 (+ 1851us) server_negotiation.cc:407] Connection header received
0812 17:53:07.316000 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.316004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.316066 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.316119 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.321085 (+ 4966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51364: BlockingRecv error: recv got EOF from 127.0.0.1:51364 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 17:53:07.326588 16328 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.325063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37742 (local address 127.13.38.61:32931)
0812 17:53:07.325221 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.325226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.325527 (+ 301us) server_negotiation.cc:407] Connection header received
0812 17:53:07.325706 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.325712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.325825 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.325937 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.326426 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37742: BlockingRecv error: recv got EOF from 127.0.0.1:37742 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 17:53:07.330776 16328 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.329651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37752 (local address 127.13.38.61:32931)
0812 17:53:07.329798 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.329805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.329825 (+ 20us) server_negotiation.cc:407] Connection header received
0812 17:53:07.329896 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.329913 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.329986 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.330128 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.330607 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37752: BlockingRecv error: recv got EOF from 127.0.0.1:37752 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 17:53:07.336654 16328 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.335550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37758 (local address 127.13.38.61:32931)
0812 17:53:07.335708 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.335715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.335734 (+ 19us) server_negotiation.cc:407] Connection header received
0812 17:53:07.335805 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.335810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.335880 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.335984 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.336484 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37758: BlockingRecv error: recv got EOF from 127.0.0.1:37758 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 17:53:07.344835 16328 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.343106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37774 (local address 127.13.38.61:32931)
0812 17:53:07.343351 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.343357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.343725 (+ 368us) server_negotiation.cc:407] Connection header received
0812 17:53:07.343954 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.343959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.344041 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.344150 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.344659 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37774: BlockingRecv error: recv got EOF from 127.0.0.1:37774 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
W20260812 17:53:07.356519 16328 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.354887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37786 (local address 127.13.38.61:32931)
0812 17:53:07.355045 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.355050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.355451 (+ 401us) server_negotiation.cc:407] Connection header received
0812 17:53:07.355619 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.355624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.355705 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.355809 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.356344 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37786: BlockingRecv error: recv got EOF from 127.0.0.1:37786 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 17:53:07.376379 16328 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.374757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37800 (local address 127.13.38.61:32931)
0812 17:53:07.374921 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.374927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.375187 (+ 260us) server_negotiation.cc:407] Connection header received
0812 17:53:07.375358 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.375364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.375442 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.375588 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.376157 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37800: BlockingRecv error: recv got EOF from 127.0.0.1:37800 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 17:53:07.412073 16328 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.410600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37810 (local address 127.13.38.61:32931)
0812 17:53:07.410740 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.410747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.410956 (+ 209us) server_negotiation.cc:407] Connection header received
0812 17:53:07.411159 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.411166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.411220 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.411323 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.411900 (+ 577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37810: BlockingRecv error: recv got EOF from 127.0.0.1:37810 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 17:53:07.480216 16328 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.478573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37822 (local address 127.13.38.61:32931)
0812 17:53:07.478760 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.478765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.479017 (+ 252us) server_negotiation.cc:407] Connection header received
0812 17:53:07.479244 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.479247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.479303 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.479388 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.480084 (+ 696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37822: BlockingRecv error: recv got EOF from 127.0.0.1:37822 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260812 17:53:07.611992 16328 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.610677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.13.38.61:32931)
0812 17:53:07.610852 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.610857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.610988 (+ 131us) server_negotiation.cc:407] Connection header received
0812 17:53:07.611178 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.611181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.611243 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.611341 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.611863 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37832: BlockingRecv error: recv got EOF from 127.0.0.1:37832 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 17:53:07.871685 16328 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:07.870468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37834 (local address 127.13.38.61:32931)
0812 17:53:07.870619 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:07.870623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:07.870640 (+ 17us) server_negotiation.cc:407] Connection header received
0812 17:53:07.870764 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:07.870766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:07.870814 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:07.870891 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:07.871523 (+ 632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37834: BlockingRecv error: recv got EOF from 127.0.0.1:37834 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 17:53:08.051177 16469 catalog_manager.cc:1259] Loaded cluster ID: f99b8c984f6041159cf227b31da2bd99
I20260812 17:53:08.051312 16469 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c: loading cluster ID for follower catalog manager: success
I20260812 17:53:08.053170 16469 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c: acquiring CA information for follower catalog manager: success
I20260812 17:53:08.054430 16469 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cc63c0536baa470fbf77c9a499ce8e9c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 17:53:08.073331 16487 catalog_manager.cc:1259] Loaded cluster ID: f99b8c984f6041159cf227b31da2bd99
I20260812 17:53:08.073431 16487 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d: loading cluster ID for follower catalog manager: success
I20260812 17:53:08.075155 16487 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d: acquiring CA information for follower catalog manager: success
I20260812 17:53:08.076391 16487 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a1a74f93655d4be2ab9155a63a2c264d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 17:53:08.390704 16641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:08.386095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37836 (local address 127.13.38.61:32931)
0812 17:53:08.386367 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:08.386371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:08.386548 (+ 177us) server_negotiation.cc:407] Connection header received
0812 17:53:08.386652 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:08.386654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:08.386806 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:08.386904 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:53:08.387250 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:08.387844 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:08.388857 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:08.389092 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:08.389181 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:08.389633 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:53:08.390160 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:53:08.390272 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:08.390405 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 17:53:08.390469 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":111,"threads_started":1}
I20260812 17:53:08.397842 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:08.394174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51372 (local address 127.13.38.60:35743)
0812 17:53:08.394434 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:08.394438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:08.394450 (+ 12us) server_negotiation.cc:407] Connection header received
0812 17:53:08.394531 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:08.394533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:08.394648 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:08.394752 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:53:08.394976 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:08.395494 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:08.396373 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:08.396608 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:08.396701 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:08.396981 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:53:08.397422 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:53:08.397522 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:08.397575 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 17:53:08.397624 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":107,"threads_started":1}
I20260812 17:53:08.405658 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:08.401507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51376 (local address 127.13.38.60:35743)
0812 17:53:08.401637 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:08.401640 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:08.402240 (+ 600us) server_negotiation.cc:407] Connection header received
0812 17:53:08.402298 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:08.402301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:08.402351 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:08.402423 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:53:08.402751 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:08.403423 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:08.404382 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:08.404600 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:08.404711 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:08.405080 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:53:08.405167 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:53:08.405245 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:08.405432 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 17:53:08.405489 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 17:53:08.405893 16641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:08.401068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37846 (local address 127.13.38.61:32931)
0812 17:53:08.401214 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:08.401219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:08.402199 (+ 980us) server_negotiation.cc:407] Connection header received
0812 17:53:08.402426 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:08.402429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:08.402474 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:08.402557 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:53:08.402855 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:08.403460 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:08.404684 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:08.405001 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:08.405105 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:08.405372 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:53:08.405446 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:53:08.405521 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:08.405692 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 17:53:08.405764 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 17:53:08.405907 16665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:08.401052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.13.38.62:42429)
0812 17:53:08.401292 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:08.401297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:08.401461 (+ 164us) server_negotiation.cc:407] Connection header received
0812 17:53:08.401649 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:08.401651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:08.401778 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:08.401877 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:53:08.402132 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:08.402996 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:08.404190 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:08.404543 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:08.404648 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:08.405100 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:53:08.405486 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:53:08.405571 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:08.405724 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 17:53:08.405780 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":68,"threads_started":1}
I20260812 17:53:08.409884 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 16094
I20260812 17:53:08.428541 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 16215
I20260812 17:53:08.444474 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 16334
2026-08-12T17:53:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3562 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 17:53:08 dist-test-slave-sb2z krb5kdc[16673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 17:53:08 dist-test-slave-sb2z krb5kdc[16673](info): set up 2 sockets
Aug 12 17:53:08 dist-test-slave-sb2z krb5kdc[16673](info): commencing operation
krb5kdc: starting...
W20260812 17:53:09.755257 13464 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/test-user.keytab.
Aug 12 17:53:09 dist-test-slave-sb2z krb5kdc[16673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T17:53:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T17:53:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.38.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
Entry for principal kudu/127.13.38.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab.
I20260812 17:53:09.904630 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:33233
--webserver_interface=127.13.38.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:46819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:33233,127.13.38.61:44347,127.13.38.60:45653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 17:53:10.030368 16692 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 17:53:10.030714 16692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:53:10.030738 16692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:53:10.030844 16692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:53:10.035693 16692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:53:10.035789 16692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:53:10.035813 16692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:53:10.035835 16692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:53:10.035916 16692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:53:10.042289 16692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:46819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:33233,127.13.38.61:44347,127.13.38.60:45653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.62.keytab
--principal=kudu/127.13.38.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.62:33233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.38.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:53:10.043637 16692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:53:10.044770 16692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:53:10.054335 16707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:10.054329 16702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:10.054292 16704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:53:10.054323 16692 server_base.cc:1034] running on GCE node
I20260812 17:53:10.055265 16692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:53:10.056567 16692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:53:10.057822 16692 hybrid_clock.cc:648] HybridClock initialized: now 1786557190057790 us; error 49 us; skew 500 ppm
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557190, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:53:10.067580 16692 init.cc:377] Logged in from keytab as kudu/127.13.38.62@KRBTEST.COM (short username kudu)
I20260812 17:53:10.068683 16692 webserver.cc:460] Webserver started at http://127.13.38.62:34257/ using document root <none> and password file <none>
I20260812 17:53:10.069334 16692 fs_manager.cc:359] Metadata directory not provided
I20260812 17:53:10.069444 16692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:53:10.069782 16692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:53:10.072207 16692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/data/instance:
uuid: "375c0993ca284f149b397c596d75a347"
format_stamp: "Formatted at 2026-08-12 17:53:10 on dist-test-slave-sb2z"
server_key: "c5c61d028b6ae7f60644e98e50c8a9cf"
server_key_iv: "dbd00b19aece1ba81e8594f4968dcffc"
server_key_version: "encryptionkey@0"
I20260812 17:53:10.072861 16692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/wal/instance:
uuid: "375c0993ca284f149b397c596d75a347"
format_stamp: "Formatted at 2026-08-12 17:53:10 on dist-test-slave-sb2z"
server_key: "c5c61d028b6ae7f60644e98e50c8a9cf"
server_key_iv: "dbd00b19aece1ba81e8594f4968dcffc"
server_key_version: "encryptionkey@0"
I20260812 17:53:10.077320 16692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 17:53:10.080255 16718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:53:10.081677 16692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 17:53:10.081851 16692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/wal
uuid: "375c0993ca284f149b397c596d75a347"
format_stamp: "Formatted at 2026-08-12 17:53:10 on dist-test-slave-sb2z"
server_key: "c5c61d028b6ae7f60644e98e50c8a9cf"
server_key_iv: "dbd00b19aece1ba81e8594f4968dcffc"
server_key_version: "encryptionkey@0"
I20260812 17:53:10.082005 16692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:53:10.104650 16692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:53:10.108390 16692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:53:10.108639 16692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:53:10.116739 16692 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.62:33233
I20260812 17:53:10.117127 16805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.62:33233 every 8 connection(s)
I20260812 17:53:10.117831 16692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/data/info.pb
I20260812 17:53:10.121014 16807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:53:10.124212 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 16692
I20260812 17:53:10.124368 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-0/wal/instance
I20260812 17:53:10.124675 13464 external_mini_cluster.cc:1428] Setting key efec3728a140cddc2c6ec3a47ae283e5
I20260812 17:53:10.125301 16807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.38.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.61@KRBTEST.COM" created.
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557190, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:53:10.145561 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.127751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:33233 (local address 127.0.0.1:55706)
0812 17:53:10.128286 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.128341 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:10.129288 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:10.130270 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:10.130275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:10.130661 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:10.130926 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.130940 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.132966 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.132970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.133823 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.133835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.134839 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.134842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.135024 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.135677 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:10.135694 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:10.139588 (+ 3894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:10.142264 (+ 2676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.142273 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.142283 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.142558 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.142973 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.142975 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.142977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.143095 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.144018 (+ 923us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:10.144024 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:10.144240 (+ 216us) client_negotiation.cc:769] Sending connection context
0812 17:53:10.144423 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 17:53:10.144721 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":151,"threads_started":1}
I20260812 17:53:10.145567 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.127752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55706 (local address 127.13.38.62:33233)
0812 17:53:10.128356 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.128371 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.128433 (+ 62us) server_negotiation.cc:407] Connection header received
0812 17:53:10.129684 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.129695 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.130003 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.130234 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.131113 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.132749 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.133975 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.134635 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.134812 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.139805 (+ 4993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.139811 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.139825 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.139851 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.142107 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.142692 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.142702 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.142707 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.142867 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.143196 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.143203 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.143207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.143875 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.144044 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.144558 (+ 514us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.144874 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":465,"thread_start_us":173,"threads_started":1}
I20260812 17:53:10.148167 16807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:10.149010 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.148727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:44347 (local address 127.0.0.1:43752)
0812 17:53:10.148860 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.148943 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:44347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 17:53:10.149281 16721 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.38.61:44347: connect: Connection refused (error 111)
I20260812 17:53:10.150213 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.149976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:44347 (local address 127.0.0.1:43754)
0812 17:53:10.150109 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.150161 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:44347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 17:53:10.150365 16807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.61:44347: Network error: Client connection negotiation failed: client connection to 127.13.38.61:44347: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.61.keytab.
Entry for principal kudu/127.13.38.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.61.keytab.
I20260812 17:53:10.160768 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.61:44347
--webserver_interface=127.13.38.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.61.keytab
--principal=kudu/127.13.38.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:46819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:33233,127.13.38.61:44347,127.13.38.60:45653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:53:10.199666 16807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } attempt: 1
I20260812 17:53:10.200716 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.200360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:44347 (local address 127.0.0.1:43762)
0812 17:53:10.200559 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.200631 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:44347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 17:53:10.201661 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.201411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:44347 (local address 127.0.0.1:43778)
0812 17:53:10.201540 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.201602 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:44347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 17:53:10.201807 16807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.61:44347: Network error: Client connection negotiation failed: client connection to 127.13.38.61:44347: connect: Connection refused (error 111)
I20260812 17:53:10.269984 16807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } attempt: 2
I20260812 17:53:10.270879 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.270560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:44347 (local address 127.0.0.1:43788)
0812 17:53:10.270731 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.270802 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:44347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 17:53:10.271674 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.271417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:44347 (local address 127.0.0.1:43798)
0812 17:53:10.271559 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.271614 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.61:44347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 17:53:10.271818 16807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.61:44347: Network error: Client connection negotiation failed: client connection to 127.13.38.61:44347: connect: Connection refused (error 111)
W20260812 17:53:10.286165 16820 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 17:53:10.286594 16820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:53:10.286671 16820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:53:10.286755 16820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:53:10.291163 16820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:53:10.291273 16820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:53:10.291316 16820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:53:10.291369 16820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:53:10.291409 16820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:53:10.297132 16820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:46819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:33233,127.13.38.61:44347,127.13.38.60:45653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.61.keytab
--principal=kudu/127.13.38.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.61:44347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.38.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:53:10.298398 16820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:53:10.299582 16820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:53:10.309285 16827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:10.309276 16826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:53:10.309427 16820 server_base.cc:1034] running on GCE node
W20260812 17:53:10.309278 16830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:53:10.310299 16820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:53:10.311627 16820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:53:10.312834 16820 hybrid_clock.cc:648] HybridClock initialized: now 1786557190312796 us; error 53 us; skew 500 ppm
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557190, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:53:10.316092 16820 init.cc:377] Logged in from keytab as kudu/127.13.38.61@KRBTEST.COM (short username kudu)
I20260812 17:53:10.317168 16820 webserver.cc:460] Webserver started at http://127.13.38.61:35819/ using document root <none> and password file <none>
I20260812 17:53:10.317731 16820 fs_manager.cc:359] Metadata directory not provided
I20260812 17:53:10.317792 16820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:53:10.317981 16820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:53:10.320226 16820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/data/instance:
uuid: "7a03c0b9d2b54f7c87451aed45f973fc"
format_stamp: "Formatted at 2026-08-12 17:53:10 on dist-test-slave-sb2z"
server_key: "18bdcd8bbb665c879a4a79ca0a6ec97f"
server_key_iv: "97aba31367ef66f765388891b095aba9"
server_key_version: "encryptionkey@0"
I20260812 17:53:10.320770 16820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/wal/instance:
uuid: "7a03c0b9d2b54f7c87451aed45f973fc"
format_stamp: "Formatted at 2026-08-12 17:53:10 on dist-test-slave-sb2z"
server_key: "18bdcd8bbb665c879a4a79ca0a6ec97f"
server_key_iv: "97aba31367ef66f765388891b095aba9"
server_key_version: "encryptionkey@0"
I20260812 17:53:10.324719 16820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 17:53:10.327520 16850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:53:10.328881 16820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 17:53:10.329026 16820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/wal
uuid: "7a03c0b9d2b54f7c87451aed45f973fc"
format_stamp: "Formatted at 2026-08-12 17:53:10 on dist-test-slave-sb2z"
server_key: "18bdcd8bbb665c879a4a79ca0a6ec97f"
server_key_iv: "97aba31367ef66f765388891b095aba9"
server_key_version: "encryptionkey@0"
I20260812 17:53:10.329162 16820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:53:10.347805 16820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:53:10.351321 16820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:53:10.351608 16820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:53:10.359601 16820 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.61:44347
I20260812 17:53:10.359929 16947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.61:44347 every 8 connection(s)
I20260812 17:53:10.360770 16820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/data/info.pb
I20260812 17:53:10.363003 16807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } attempt: 3
I20260812 17:53:10.364174 16948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:53:10.368330 16948 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:10.369565 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 16820
I20260812 17:53:10.369720 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-1/wal/instance
I20260812 17:53:10.370117 13464 external_mini_cluster.cc:1428] Setting key 3297e7a1914c76adb06053e02044e355
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557190, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for kudu/127.13.38.61@KRBTEST.COM
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557190, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.61@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:53:10.378453 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.363692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:44347 (local address 127.0.0.1:43802)
0812 17:53:10.363849 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.363869 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:10.364015 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:10.366482 (+ 2467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:10.366486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:10.366723 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:10.366874 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.366883 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.368413 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.368419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.369073 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.369085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.370278 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.370283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.370435 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.371089 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:10.371112 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:10.372993 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:10.375933 (+ 2940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.375938 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.375941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.376177 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.376695 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.376698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.376700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.376817 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.377782 (+ 965us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:10.377786 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:10.377913 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 17:53:10.378047 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 17:53:10.378215 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 17:53:10.380718 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.364457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43802 (local address 127.13.38.61:44347)
0812 17:53:10.364969 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.364987 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.365029 (+ 42us) server_negotiation.cc:407] Connection header received
0812 17:53:10.365772 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.365790 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.366161 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.366561 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.367055 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.368259 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.369228 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.370100 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.370793 (+ 693us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.373222 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.373228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.373239 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.373268 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.375803 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.376340 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.376350 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.376360 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.376555 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.376947 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.376954 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.376959 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.377626 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.377825 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.378973 (+ 1148us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.379375 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":160,"threads_started":1}
I20260812 17:53:10.382819 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.370252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55710 (local address 127.13.38.62:33233)
0812 17:53:10.370405 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.370410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.370861 (+ 451us) server_negotiation.cc:407] Connection header received
0812 17:53:10.371108 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.371114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.371183 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.371280 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.372159 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.372716 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.374146 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.375427 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.375635 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.378593 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.378597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.378600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.378628 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.380604 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.381246 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.381251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.381252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.381348 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.381687 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.381690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.381692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.381954 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.382039 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.382515 (+ 476us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.382643 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 17:53:10.382844 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.370237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:33233 (local address 127.0.0.1:55710)
0812 17:53:10.370735 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.370786 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:10.370951 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:10.371363 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:10.371368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:10.371872 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:10.372019 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.372025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.372895 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.372900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.373995 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.374010 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.375695 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.375703 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.375946 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.376650 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:10.376676 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:10.378412 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:10.380749 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.380758 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.380765 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.381113 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.381476 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.381479 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.381481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.381576 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.382130 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:10.382136 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:10.382273 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 17:53:10.382433 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 17:53:10.382643 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.13.38.60@KRBTEST.COM; defaulting to no policy
I20260812 17:53:10.383534 16807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:10.384394 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.384078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39410)
0812 17:53:10.384258 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.384321 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 17:53:10.384748 16948 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:10.385278 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.385006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39424)
0812 17:53:10.385152 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.385200 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260812 17:53:10.385435 16807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:45653: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.38.60@KRBTEST.COM" created.
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557190, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.61@KRBTEST.COM for kudu/127.13.38.61@KRBTEST.COM
I20260812 17:53:10.394625 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.385442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43812 (local address 127.13.38.61:44347)
0812 17:53:10.385604 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.385608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.385623 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:53:10.385737 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.385739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.385807 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.385900 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.386237 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.387061 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.387883 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.388282 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.388461 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.390870 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.390873 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.390874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.390896 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.392720 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.393253 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.393256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.393257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.393333 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.393665 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.393668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.393669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.393857 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.393934 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.394283 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.394486 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":120,"server-negotiator.queue_time_us":85,"spinlock_wait_cycles":41216}
I20260812 17:53:10.394701 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.385351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:44347 (local address 127.0.0.1:43812)
0812 17:53:10.385538 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.385552 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:10.385645 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:10.385889 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:10.385891 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:10.386032 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:10.386142 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.386147 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.387192 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.387194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.387773 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.387782 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.388395 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.388398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.388509 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.389042 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:10.389061 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:10.390734 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:10.392856 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.392862 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.392867 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.393150 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.393434 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.393440 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.393445 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.393553 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.393961 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:10.393967 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:10.394096 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 17:53:10.394223 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 17:53:10.394417 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":17}
I20260812 17:53:10.395488 16948 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:10.396386 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.396053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39438)
0812 17:53:10.396202 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.396301 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 17:53:10.397020 16854 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
I20260812 17:53:10.397984 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.397787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39440)
0812 17:53:10.397888 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.397933 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 17:53:10.398151 16948 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:45653: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.38.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.60.keytab.
Entry for principal kudu/127.13.38.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.60.keytab.
I20260812 17:53:10.406606 13464 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
/tmp/dist-test-taskuCZvAR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.60:45653
--webserver_interface=127.13.38.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.60.keytab
--principal=kudu/127.13.38.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.38.20:46819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.38.62:33233,127.13.38.61:44347,127.13.38.60:45653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 17:53:10.416620 16807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } attempt: 1
I20260812 17:53:10.417629 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.417306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39448)
0812 17:53:10.417480 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.417545 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 17:53:10.418449 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.418212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39460)
0812 17:53:10.418333 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.418388 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 17:53:10.418588 16807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:45653: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
I20260812 17:53:10.447479 16948 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } attempt: 1
I20260812 17:53:10.448642 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.448253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39462)
0812 17:53:10.448460 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.448539 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 17:53:10.449515 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.449258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39474)
0812 17:53:10.449386 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.449444 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 17:53:10.449685 16948 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:45653: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
I20260812 17:53:10.493767 16807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } attempt: 2
I20260812 17:53:10.494755 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.494444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39486)
0812 17:53:10.494612 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.494677 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 17:53:10.495748 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.495462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39494)
0812 17:53:10.495644 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.495691 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260812 17:53:10.495934 16807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:45653: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
I20260812 17:53:10.517854 16948 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } attempt: 2
I20260812 17:53:10.518802 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.518458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39500)
0812 17:53:10.518633 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.518706 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 17:53:10.519680 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.519405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39510)
0812 17:53:10.519550 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.519611 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 17:53:10.519882 16948 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:45653: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
W20260812 17:53:10.528429 16959 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 17:53:10.528752 16959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 17:53:10.528810 16959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 17:53:10.528903 16959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 17:53:10.533206 16959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 17:53:10.533279 16959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 17:53:10.533299 16959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 17:53:10.533354 16959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 17:53:10.533386 16959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 17:53:10.539808 16959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.38.20:46819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.38.62:33233,127.13.38.61:44347,127.13.38.60:45653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/krb5kdc/kudu_127.13.38.60.keytab
--principal=kudu/127.13.38.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.38.60:45653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.38.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 16:58:54 UTC on ccycloud.quasar-ctecla.root.comops.site
build id 4583009
I20260812 17:53:10.541121 16959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 17:53:10.542239 16959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 17:53:10.551618 16973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 17:53:10.551625 16970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:53:10.551898 16959 server_base.cc:1034] running on GCE node
W20260812 17:53:10.551626 16968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 17:53:10.552717 16959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 17:53:10.554065 16959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 17:53:10.555236 16959 hybrid_clock.cc:648] HybridClock initialized: now 1786557190555194 us; error 58 us; skew 500 ppm
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557190, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 17:53:10.558336 16959 init.cc:377] Logged in from keytab as kudu/127.13.38.60@KRBTEST.COM (short username kudu)
I20260812 17:53:10.559310 16959 webserver.cc:460] Webserver started at http://127.13.38.60:45873/ using document root <none> and password file <none>
I20260812 17:53:10.559998 16959 fs_manager.cc:359] Metadata directory not provided
I20260812 17:53:10.560067 16959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 17:53:10.560261 16959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 17:53:10.562445 16959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/data/instance:
uuid: "d4b2cece15044b47b224bd75f7a49830"
format_stamp: "Formatted at 2026-08-12 17:53:10 on dist-test-slave-sb2z"
server_key: "be6b62425c2cd036dd6a012a39968cd4"
server_key_iv: "eef81388cc613b725bbfd4c12455e3e6"
server_key_version: "encryptionkey@0"
I20260812 17:53:10.563009 16959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/wal/instance:
uuid: "d4b2cece15044b47b224bd75f7a49830"
format_stamp: "Formatted at 2026-08-12 17:53:10 on dist-test-slave-sb2z"
server_key: "be6b62425c2cd036dd6a012a39968cd4"
server_key_iv: "eef81388cc613b725bbfd4c12455e3e6"
server_key_version: "encryptionkey@0"
I20260812 17:53:10.567056 16959 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 17:53:10.570155 16983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 17:53:10.571753 16959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 17:53:10.571885 16959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/data,/tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/wal
uuid: "d4b2cece15044b47b224bd75f7a49830"
format_stamp: "Formatted at 2026-08-12 17:53:10 on dist-test-slave-sb2z"
server_key: "be6b62425c2cd036dd6a012a39968cd4"
server_key_iv: "eef81388cc613b725bbfd4c12455e3e6"
server_key_version: "encryptionkey@0"
I20260812 17:53:10.572032 16959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 17:53:10.595121 16807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } attempt: 3
I20260812 17:53:10.596110 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.595780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39522)
0812 17:53:10.595936 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.596013 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 17:53:10.596849 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.596639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39532)
0812 17:53:10.596750 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.596798 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 17:53:10.596984 16807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:45653: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
I20260812 17:53:10.611200 16948 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } attempt: 3
I20260812 17:53:10.612078 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.611774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39534)
0812 17:53:10.611922 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.611999 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 17:53:10.612862 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.612599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39546)
0812 17:53:10.612752 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.612804 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260812 17:53:10.613026 16948 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.38.60:45653: Network error: Client connection negotiation failed: client connection to 127.13.38.60:45653: connect: Connection refused (error 111)
I20260812 17:53:10.613989 16959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 17:53:10.617384 16959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 17:53:10.617610 16959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 17:53:10.625416 16959 rpc_server.cc:307] RPC server started. Bound to: 127.13.38.60:45653
I20260812 17:53:10.625741 17068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.38.60:45653 every 8 connection(s)
I20260812 17:53:10.626504 16959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/data/info.pb
I20260812 17:53:10.629899 17069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 17:53:10.634519 17069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:10.635106 13464 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu as pid 16959
I20260812 17:53:10.635241 13464 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskuCZvAR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786556880371321-13464-0/minicluster-data/master-2/wal/instance
I20260812 17:53:10.635600 13464 external_mini_cluster.cc:1428] Setting key 944148687606fa1cf7402b0013bca6fe
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557190, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.60@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.60@KRBTEST.COM
I20260812 17:53:10.649587 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.636949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55722 (local address 127.13.38.62:33233)
0812 17:53:10.637080 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.637084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.638068 (+ 984us) server_negotiation.cc:407] Connection header received
0812 17:53:10.639075 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.639078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.639143 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.639226 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.640248 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.640763 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.641850 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.642103 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.642222 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.645378 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.645382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.645385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.645425 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.647124 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.647782 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.647786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.647788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.647879 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.648243 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.648246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.648248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.648456 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.648573 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.649249 (+ 676us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.649395 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 17:53:10.650563 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.637334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.62:33233 (local address 127.0.0.1:55722)
0812 17:53:10.637884 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.637942 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:10.638829 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:10.639302 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:10.639308 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:10.639773 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:10.640057 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.640072 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.640902 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.640907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.641706 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.641716 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.642284 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.642288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.642474 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.643172 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:10.643196 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:10.645195 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:10.647267 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.647279 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.647291 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.647665 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.647978 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.647982 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.647984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.648135 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.648600 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:10.648607 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:10.648908 (+ 301us) client_negotiation.cc:769] Sending connection context
0812 17:53:10.649226 (+ 318us) client_negotiation.cc:240] Negotiation successful
0812 17:53:10.649627 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":38,"spinlock_wait_cycles":896,"thread_start_us":130,"threads_started":1}
I20260812 17:53:10.651167 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.638153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39548 (local address 127.13.38.60:45653)
0812 17:53:10.638431 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.638441 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.638475 (+ 34us) server_negotiation.cc:407] Connection header received
0812 17:53:10.638856 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.638889 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.639128 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.639363 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.640404 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.641325 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.642195 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.642567 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.642689 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.645587 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.645594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.645606 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.645627 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.647501 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.648023 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.648030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.648031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.648138 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.648453 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.648457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.648458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.648895 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.649182 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.649373 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.650962 (+ 1589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1405,"server-negotiator.queue_time_us":170,"thread_start_us":92,"threads_started":1}
I20260812 17:53:10.652673 17069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557190, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.60@KRBTEST.COM for kudu/127.13.38.61@KRBTEST.COM
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.62@KRBTEST.COM
I20260812 17:53:10.662441 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.653308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.61:44347 (local address 127.0.0.1:43826)
0812 17:53:10.653486 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.653501 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:10.653583 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:10.653841 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:10.653844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:10.653994 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:10.654105 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.654110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.654942 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.654948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.655501 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.655511 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.655997 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.656001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.656107 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.656679 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:10.656691 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:10.658169 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:10.660636 (+ 2467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.660639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.660640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.660984 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.661323 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.661326 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.661327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.661408 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.661877 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:10.661880 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:10.661986 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 17:53:10.662108 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 17:53:10.662264 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260812 17:53:10.662441 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.653432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43826 (local address 127.13.38.61:44347)
0812 17:53:10.653596 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.653600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.653618 (+ 18us) server_negotiation.cc:407] Connection header received
0812 17:53:10.653676 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.653679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.653729 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.653816 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.654228 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.654819 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.655664 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.655884 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.655965 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.658329 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.658334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.658337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.658370 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.660497 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.661109 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.661113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.661115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.661215 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.661511 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.661514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.661516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.661790 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.661884 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.662162 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.662280 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 17:53:10.663213 17069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } has no permanent_uuid. Determining permanent_uuid...
I20260812 17:53:10.663884 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.653971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55732 (local address 127.13.38.62:33233)
0812 17:53:10.654145 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.654148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.654180 (+ 32us) server_negotiation.cc:407] Connection header received
0812 17:53:10.654219 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.654222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.654265 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.654344 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.655030 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.655496 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.656492 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.656806 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.656889 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.659876 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.659881 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.659885 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.659917 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.661688 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.662337 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.662342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.662344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.662463 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.662852 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.662855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.662857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.663065 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.663337 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.663545 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.663680 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557190, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.60@KRBTEST.COM for kudu/127.13.38.60@KRBTEST.COM
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.38.61@KRBTEST.COM
I20260812 17:53:10.674016 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.663815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39558)
0812 17:53:10.663938 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.663954 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:10.664031 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:10.664421 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:10.664423 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:10.664601 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:10.664699 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.664705 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.665788 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.665793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.666779 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.666790 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.667482 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.667485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.667651 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.668329 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:10.668345 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:10.670013 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:10.672210 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.672213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.672215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.672408 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.672816 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.672819 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.672820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.672959 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.673567 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:10.673569 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:10.673660 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 17:53:10.673763 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 17:53:10.673879 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 17:53:10.674381 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.664040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39558 (local address 127.13.38.60:45653)
0812 17:53:10.664150 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.664154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.664167 (+ 13us) server_negotiation.cc:407] Connection header received
0812 17:53:10.664198 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.664200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.664231 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.664321 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.664824 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.665575 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.666919 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.667291 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.667406 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.670169 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.670172 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.670174 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.670195 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.672060 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.672542 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.672546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.672548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.672667 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.673124 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.673128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.673129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.673438 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.673563 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.673873 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.674190 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 17:53:10.676307 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.665312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43840 (local address 127.13.38.61:44347)
0812 17:53:10.665466 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.665470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.665485 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:53:10.665605 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.665617 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.665686 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.665766 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.666685 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.667439 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.668701 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.668918 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.669086 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.672181 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.672189 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.672195 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.672229 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.674229 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.674804 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.674807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.674809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.674894 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.675330 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.675333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.675334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.675619 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.675876 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.676004 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.676158 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 17:53:10.679430 17069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830: Bootstrap starting.
I20260812 17:53:10.680992 13464 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 17:53:10.682651 17069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830: Neither blocks nor log segments found. Creating new log.
I20260812 17:53:10.683638 17069 log.cc:824] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830: Log is configured to *not* fsync() on all Append() calls
I20260812 17:53:10.682399 16763 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "375c0993ca284f149b397c596d75a347"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55732
I20260812 17:53:10.686120 17069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830: No bootstrap required, opened a new log
I20260812 17:53:10.690529 17069 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } }
I20260812 17:53:10.690860 17069 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4b2cece15044b47b224bd75f7a49830, State: Initialized, Role: FOLLOWER
I20260812 17:53:10.691445 17069 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } }
I20260812 17:53:10.692142 17077 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } } }
I20260812 17:53:10.692338 17077 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:53:10.693044 17069 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 17:53:10.698820 17094 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:53:10.698968 17094 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:53:10.697407 16959 master_runner.cc:186] Error getting master registration for 127.13.38.62:33233: OK, instance_id { permanent_uuid: "375c0993ca284f149b397c596d75a347" instance_seqno: 1786557190114540 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 17:53:10.701160 16959 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 17:53:10.756218 16948 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } attempt: 4
I20260812 17:53:10.761233 16807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } attempt: 4
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557190, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.61@KRBTEST.COM for kudu/127.13.38.60@KRBTEST.COM
I20260812 17:53:10.767282 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.756962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39564)
0812 17:53:10.757139 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.757159 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:10.757307 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:10.757652 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:10.757655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:10.757893 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:10.758014 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.758020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.759120 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.759123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.760015 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.760028 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.760671 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.760675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.760775 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.761446 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:10.761463 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:10.763106 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:10.765307 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.765326 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.765328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.765638 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.765990 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.765993 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.765995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.766114 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.766705 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:10.766708 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:10.766818 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 17:53:10.766918 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 17:53:10.767079 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
Aug 12 17:53:10 dist-test-slave-sb2z krb5kdc[16673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786557190, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.38.62@KRBTEST.COM for kudu/127.13.38.60@KRBTEST.COM
I20260812 17:53:10.767952 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.757126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.13.38.60:45653)
0812 17:53:10.757276 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.757281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.757300 (+ 19us) server_negotiation.cc:407] Connection header received
0812 17:53:10.757434 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.757438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.757498 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.757603 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.758165 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.758988 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.760158 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.760534 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.760654 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.763262 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.763266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.763269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.763299 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.765137 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.765756 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.765759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.765761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.765877 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.766234 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.766238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.766240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.766554 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.766652 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.767667 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.767798 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 17:53:10.772027 16948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc: Bootstrap starting.
I20260812 17:53:10.772110 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.761757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.38.60:45653 (local address 127.0.0.1:39578)
0812 17:53:10.761897 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 17:53:10.761915 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 17:53:10.762008 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 17:53:10.762647 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 17:53:10.762651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 17:53:10.762870 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 17:53:10.762979 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.762985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.764153 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.764159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.765063 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 17:53:10.765079 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.765775 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.765779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 17:53:10.765877 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.766456 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 17:53:10.766473 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 17:53:10.768121 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 17:53:10.770340 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.770344 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.770346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.770666 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.770983 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 17:53:10.770986 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 17:53:10.770988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 17:53:10.771085 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 17:53:10.771554 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 17:53:10.771557 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 17:53:10.771670 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 17:53:10.771781 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 17:53:10.771934 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 17:53:10.772248 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.761989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39578 (local address 127.13.38.60:45653)
0812 17:53:10.762397 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.762408 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.762423 (+ 15us) server_negotiation.cc:407] Connection header received
0812 17:53:10.762460 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.762464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.762513 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.762603 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.763116 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.763985 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.765261 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.765635 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.766161 (+ 526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.768285 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.768289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.768291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.768317 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.770207 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.770788 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.770792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.770793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.770891 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.771188 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.771192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.771193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.771434 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.771560 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.771876 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.772009 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":168,"threads_started":1}
I20260812 17:53:10.775857 16948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc: Neither blocks nor log segments found. Creating new log.
I20260812 17:53:10.775882 16807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347: Bootstrap starting.
I20260812 17:53:10.776953 16948 log.cc:824] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc: Log is configured to *not* fsync() on all Append() calls
I20260812 17:53:10.778857 16807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347: Neither blocks nor log segments found. Creating new log.
I20260812 17:53:10.779632 16948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc: No bootstrap required, opened a new log
I20260812 17:53:10.779814 16807 log.cc:824] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347: Log is configured to *not* fsync() on all Append() calls
I20260812 17:53:10.782307 16807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347: No bootstrap required, opened a new log
I20260812 17:53:10.783732 16948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } }
I20260812 17:53:10.783973 16948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a03c0b9d2b54f7c87451aed45f973fc, State: Initialized, Role: FOLLOWER
I20260812 17:53:10.784327 16948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } }
I20260812 17:53:10.785004 17102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } } }
I20260812 17:53:10.785398 17102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:53:10.785467 16807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } }
I20260812 17:53:10.785693 16807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375c0993ca284f149b397c596d75a347, State: Initialized, Role: FOLLOWER
I20260812 17:53:10.785832 16948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:53:10.786011 16807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } }
I20260812 17:53:10.786656 17103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } } }
I20260812 17:53:10.786901 17103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:53:10.787482 16807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 17:53:10.787595 16763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 17:53:10.787707 16763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:53:10.789022 16763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } }
I20260812 17:53:10.790010 16763 leader_election.cc:288] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [CANDIDATE]: Term 1 election: Requested vote from peers 7a03c0b9d2b54f7c87451aed45f973fc (127.13.38.61:44347), d4b2cece15044b47b224bd75f7a49830 (127.13.38.60:45653)
I20260812 17:53:10.791514 17025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "375c0993ca284f149b397c596d75a347" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d4b2cece15044b47b224bd75f7a49830"
I20260812 17:53:10.791821 17025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:53:10.793264 17025 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 375c0993ca284f149b397c596d75a347 in term 1.
W20260812 17:53:10.793548 17137 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:53:10.795855 17137 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:53:10.796197 16720 leader_election.cc:302] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 375c0993ca284f149b397c596d75a347, d4b2cece15044b47b224bd75f7a49830; no voters:
W20260812 17:53:10.796453 17119 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 17:53:10.796619 17119 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 17:53:10.794768 16900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "375c0993ca284f149b397c596d75a347" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7a03c0b9d2b54f7c87451aed45f973fc"
I20260812 17:53:10.798681 16900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 17:53:10.798931 17103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 17:53:10.799255 17103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [term 1 LEADER]: Becoming Leader. State: Replica: 375c0993ca284f149b397c596d75a347, State: Running, Role: LEADER
I20260812 17:53:10.799739 17103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } }
I20260812 17:53:10.800324 16900 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 375c0993ca284f149b397c596d75a347 in term 1.
I20260812 17:53:10.802085 17103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375c0993ca284f149b397c596d75a347. Latest consensus state: current_term: 1 leader_uuid: "375c0993ca284f149b397c596d75a347" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } } }
I20260812 17:53:10.806891 16820 master_runner.cc:263] No leader master found from master 7a03c0b9d2b54f7c87451aed45f973fc
I20260812 17:53:10.806982 16820 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 17:53:10.806955 16692 master_runner.cc:263] No leader master found from master 375c0993ca284f149b397c596d75a347
I20260812 17:53:10.808250 16692 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 17:53:10.808410 17103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [sys.catalog]: This master's current role is: LEADER
I20260812 17:53:10.809628 17149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 17:53:10.810961 17149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 17:53:10.832912 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.808072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39592 (local address 127.13.38.60:45653)
0812 17:53:10.808181 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.808185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.811299 (+ 3114us) server_negotiation.cc:407] Connection header received
0812 17:53:10.812874 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.812878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.812928 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.812981 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.815392 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.816165 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.820951 (+ 4786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.821383 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.821486 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.826343 (+ 4857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.826348 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.826351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.826390 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.828554 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.831295 (+ 2741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.831300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.831302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.831405 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.831782 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.831786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.831788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.832050 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.832178 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.832563 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.832711 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 17:53:10.833189 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.807638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43854 (local address 127.13.38.61:44347)
0812 17:53:10.807760 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.807765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.811800 (+ 4035us) server_negotiation.cc:407] Connection header received
0812 17:53:10.811999 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.812002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.812055 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.812109 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.813214 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.813997 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.819668 (+ 5671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.820045 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.820134 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.828800 (+ 8666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.828805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.828809 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.828848 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.830858 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.831503 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.831508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.831510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.831635 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.832003 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.832007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.832008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.832260 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.832385 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.832898 (+ 513us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.833030 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 17:53:10.835698 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:10.802706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55748 (local address 127.13.38.62:33233)
0812 17:53:10.802831 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.802836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.809958 (+ 7122us) server_negotiation.cc:407] Connection header received
0812 17:53:10.811568 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.811572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.811628 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.811682 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.816323 (+ 4641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.817131 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.819439 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:10.819824 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:10.819909 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:10.829169 (+ 9260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 17:53:10.829173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 17:53:10.829176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 17:53:10.829212 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 17:53:10.830981 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.834037 (+ 3056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.834042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.834044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.834160 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 17:53:10.834485 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 17:53:10.834489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 17:53:10.834491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 17:53:10.834748 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 17:53:10.834883 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:10.835246 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 17:53:10.835444 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 17:53:10.843123 16900 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [term 1 FOLLOWER]: Refusing update from remote peer 375c0993ca284f149b397c596d75a347: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:53:10.844668 17025 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [term 1 FOLLOWER]: Refusing update from remote peer 375c0993ca284f149b397c596d75a347: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 17:53:10.847820 17146 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 17:53:10.848054 17103 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 17:53:10.853277 16861 tablet.cc:2369] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:53:10.859897 17077 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375c0993ca284f149b397c596d75a347. Latest consensus state: current_term: 1 leader_uuid: "375c0993ca284f149b397c596d75a347" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } } }
I20260812 17:53:10.860080 17077 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 17:53:10.860677 16724 tablet.cc:2369] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:53:10.862207 17102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375c0993ca284f149b397c596d75a347. Latest consensus state: current_term: 1 leader_uuid: "375c0993ca284f149b397c596d75a347" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } } }
I20260812 17:53:10.862356 17102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [sys.catalog]: This master's current role is: FOLLOWER
W20260812 17:53:10.869071 16993 tablet.cc:2369] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 17:53:10.873311 17146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "375c0993ca284f149b397c596d75a347" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } } }
I20260812 17:53:10.873478 17146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [sys.catalog]: This master's current role is: LEADER
I20260812 17:53:10.873780 17152 mvcc.cc:205] Tried to move back new op lower bound from 7317738253681438720 to 7317738253520891904. Current Snapshot: MvccSnapshot[applied={T|T < 7317738253681438720}]
I20260812 17:53:10.873886 17102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "375c0993ca284f149b397c596d75a347" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } } }
I20260812 17:53:10.874006 17102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:53:10.874889 17154 mvcc.cc:205] Tried to move back new op lower bound from 7317738253681438720 to 7317738253520891904. Current Snapshot: MvccSnapshot[applied={T|T < 7317738253681438720}]
I20260812 17:53:10.876045 17146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "375c0993ca284f149b397c596d75a347" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } } }
I20260812 17:53:10.876187 17146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347 [sys.catalog]: This master's current role is: LEADER
I20260812 17:53:10.879535 17077 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "375c0993ca284f149b397c596d75a347" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375c0993ca284f149b397c596d75a347" member_type: VOTER last_known_addr { host: "127.13.38.62" port: 33233 } } peers { permanent_uuid: "7a03c0b9d2b54f7c87451aed45f973fc" member_type: VOTER last_known_addr { host: "127.13.38.61" port: 44347 } } peers { permanent_uuid: "d4b2cece15044b47b224bd75f7a49830" member_type: VOTER last_known_addr { host: "127.13.38.60" port: 45653 } } }
I20260812 17:53:10.879678 17077 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 17:53:10.883898 17153 mvcc.cc:205] Tried to move back new op lower bound from 7317738253681438720 to 7317738253520891904. Current Snapshot: MvccSnapshot[applied={T|T < 7317738253681438720}]
I20260812 17:53:10.885607 17149 catalog_manager.cc:1347] Generated new cluster ID: 048af4117e8348f8a2365e4ddc4d4916
I20260812 17:53:10.885684 17149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 17:53:10.940183 17149 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 17:53:10.941680 17149 catalog_manager.cc:1504] Loading token signing keys...
I20260812 17:53:10.956210 17149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 375c0993ca284f149b397c596d75a347: Generated new TSK 0
I20260812 17:53:10.957217 17149 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 17:53:10.987311 16814 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:10.984950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55758 (local address 127.13.38.62:33233)
0812 17:53:10.985124 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.985130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.986022 (+ 892us) server_negotiation.cc:407] Connection header received
0812 17:53:10.986248 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.986251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.986311 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.986372 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.987149 (+ 777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55758: BlockingRecv error: recv got EOF from 127.0.0.1:55758 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260812 17:53:10.988106 17097 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:10.985106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39604 (local address 127.13.38.60:45653)
0812 17:53:10.985234 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.985239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.985882 (+ 643us) server_negotiation.cc:407] Connection header received
0812 17:53:10.985967 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.985974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.986057 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.986168 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.988000 (+ 1832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39604: BlockingRecv error: recv got EOF from 127.0.0.1:39604 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 17:53:10.988996 16953 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:10.985860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43864 (local address 127.13.38.61:44347)
0812 17:53:10.986026 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.986031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.987286 (+ 1255us) server_negotiation.cc:407] Connection header received
0812 17:53:10.987346 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.987350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.987411 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.987500 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.988852 (+ 1352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43864: BlockingRecv error: recv got EOF from 127.0.0.1:43864 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260812 17:53:10.995065 16953 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:10.991444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43874 (local address 127.13.38.61:44347)
0812 17:53:10.991600 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.991605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:10.993819 (+ 2214us) server_negotiation.cc:407] Connection header received
0812 17:53:10.994111 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:10.994114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:10.994191 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:10.994325 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:10.994931 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43874: BlockingRecv error: recv got EOF from 127.0.0.1:43874 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 17:53:11.000782 16953 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:10.999859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.13.38.61:44347)
0812 17:53:10.999988 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:10.999991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:11.000004 (+ 13us) server_negotiation.cc:407] Connection header received
0812 17:53:11.000053 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:11.000055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:11.000099 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:11.000203 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:11.000676 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43888: BlockingRecv error: recv got EOF from 127.0.0.1:43888 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 17:53:11.006575 16953 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:11.005188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43890 (local address 127.13.38.61:44347)
0812 17:53:11.005320 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:11.005325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:11.005582 (+ 257us) server_negotiation.cc:407] Connection header received
0812 17:53:11.005755 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:11.005759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:11.005818 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:11.005916 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:11.006446 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43890: BlockingRecv error: recv got EOF from 127.0.0.1:43890 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 17:53:11.014407 16953 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:11.012997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43904 (local address 127.13.38.61:44347)
0812 17:53:11.013128 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:11.013133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:11.013392 (+ 259us) server_negotiation.cc:407] Connection header received
0812 17:53:11.013561 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:11.013564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:11.013615 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:11.013706 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:11.014279 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43904: BlockingRecv error: recv got EOF from 127.0.0.1:43904 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 17:53:11.026342 16953 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:11.024713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.13.38.61:44347)
0812 17:53:11.024865 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:11.024871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:11.025138 (+ 267us) server_negotiation.cc:407] Connection header received
0812 17:53:11.025334 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:11.025337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:11.025393 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:11.025500 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:11.026182 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43918: BlockingRecv error: recv got EOF from 127.0.0.1:43918 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 17:53:11.046173 16953 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:11.044810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43922 (local address 127.13.38.61:44347)
0812 17:53:11.045007 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:11.045013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:11.045171 (+ 158us) server_negotiation.cc:407] Connection header received
0812 17:53:11.045362 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:11.045366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:11.045452 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:11.045553 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:11.046060 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43922: BlockingRecv error: recv got EOF from 127.0.0.1:43922 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260812 17:53:11.082149 16953 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:11.080661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43934 (local address 127.13.38.61:44347)
0812 17:53:11.080910 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:11.080915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:11.081099 (+ 184us) server_negotiation.cc:407] Connection header received
0812 17:53:11.081280 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:11.081282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:11.081334 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:11.081434 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:11.081967 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43934: BlockingRecv error: recv got EOF from 127.0.0.1:43934 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
W20260812 17:53:11.149948 16953 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:11.148609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58876 (local address 127.13.38.61:44347)
0812 17:53:11.148763 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:11.148768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:11.148954 (+ 186us) server_negotiation.cc:407] Connection header received
0812 17:53:11.149195 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:11.149198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:11.149255 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:11.149350 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:11.149846 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58876: BlockingRecv error: recv got EOF from 127.0.0.1:58876 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 17:53:11.282359 16953 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:11.280604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58888 (local address 127.13.38.61:44347)
0812 17:53:11.281036 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:11.281039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:11.281335 (+ 296us) server_negotiation.cc:407] Connection header received
0812 17:53:11.281484 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:11.281486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:11.281543 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:11.281626 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:11.282220 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58888: BlockingRecv error: recv got EOF from 127.0.0.1:58888 (error 108)
Metrics: {"server-negotiator.queue_time_us":342}
W20260812 17:53:11.542191 16953 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 17:53:11.541061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58892 (local address 127.13.38.61:44347)
0812 17:53:11.541225 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:11.541229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:11.541245 (+ 16us) server_negotiation.cc:407] Connection header received
0812 17:53:11.541301 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:11.541303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:11.541351 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:11.541431 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 17:53:11.542081 (+ 650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58892: BlockingRecv error: recv got EOF from 127.0.0.1:58892 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 17:53:11.700470 17094 catalog_manager.cc:1259] Loaded cluster ID: 048af4117e8348f8a2365e4ddc4d4916
I20260812 17:53:11.700577 17094 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830: loading cluster ID for follower catalog manager: success
I20260812 17:53:11.702250 17094 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830: acquiring CA information for follower catalog manager: success
I20260812 17:53:11.703413 17094 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d4b2cece15044b47b224bd75f7a49830: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 17:53:11.798053 17119 catalog_manager.cc:1259] Loaded cluster ID: 048af4117e8348f8a2365e4ddc4d4916
I20260812 17:53:11.798187 17119 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc: loading cluster ID for follower catalog manager: success
I20260812 17:53:11.800071 17119 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc: acquiring CA information for follower catalog manager: success
I20260812 17:53:11.801846 17119 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7a03c0b9d2b54f7c87451aed45f973fc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 17:53:12.061081 17267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:12.056604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58896 (local address 127.13.38.61:44347)
0812 17:53:12.056898 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:12.056903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:12.057001 (+ 98us) server_negotiation.cc:407] Connection header received
0812 17:53:12.057249 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:12.057257 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:12.057403 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:12.057538 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:53:12.057844 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:12.058406 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:12.059385 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:12.059650 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:12.059771 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:12.060133 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:53:12.060610 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:53:12.060755 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:12.060822 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 17:53:12.060904 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":94,"threads_started":1}
I20260812 17:53:12.068518 17276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:12.064463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38802 (local address 127.13.38.60:45653)
0812 17:53:12.064731 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:12.064736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:12.064750 (+ 14us) server_negotiation.cc:407] Connection header received
0812 17:53:12.064931 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:12.064934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:12.065090 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:12.065228 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:53:12.065463 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:12.066021 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:12.066880 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:12.067136 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:12.067252 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:12.067634 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:53:12.068082 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:53:12.068225 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:12.068276 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 17:53:12.068335 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":81,"threads_started":1}
I20260812 17:53:12.076397 17276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:12.072176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38812 (local address 127.13.38.60:45653)
0812 17:53:12.072323 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:12.072328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:12.072469 (+ 141us) server_negotiation.cc:407] Connection header received
0812 17:53:12.072708 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:12.072714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:12.072800 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:12.072875 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:53:12.073231 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:12.073867 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:12.075116 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:12.075444 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:12.075589 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:12.075885 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:53:12.075981 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:53:12.076031 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:12.076194 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 17:53:12.076257 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 17:53:12.077148 17267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:12.072459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.13.38.61:44347)
0812 17:53:12.072613 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:12.072618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:12.072879 (+ 261us) server_negotiation.cc:407] Connection header received
0812 17:53:12.073005 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:12.073008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:12.073063 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:12.073148 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:53:12.073410 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:12.074225 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:12.075424 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:12.075773 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:12.075874 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:12.076649 (+ 775us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:53:12.076735 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:53:12.076801 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:12.076956 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 17:53:12.077022 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 17:53:12.077235 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 17:53:12.072037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40746 (local address 127.13.38.62:33233)
0812 17:53:12.072323 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0812 17:53:12.072329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 17:53:12.072566 (+ 237us) server_negotiation.cc:407] Connection header received
0812 17:53:12.072881 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 17:53:12.072885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 17:53:12.073062 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 17:53:12.073195 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 17:53:12.073562 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:12.074336 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:12.075338 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 17:53:12.075721 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 17:53:12.075841 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 17:53:12.076197 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 17:53:12.076580 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 17:53:12.076674 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 17:53:12.076984 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 17:53:12.077062 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":86,"threads_started":1}
I20260812 17:53:12.081733 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 16692
I20260812 17:53:12.100670 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 16820
I20260812 17:53:12.117010 13464 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskuCZvAR/build/debug/bin/kudu with pid 16959
2026-08-12T17:53:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3676 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (7238 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (311667 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST